Mar 23 22:17:16.222828 np0000160382 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service.
Mar 23 22:17:17.523818 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.common.config [-] Logging enabled!
Mar 23 22:17:17.524455 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev44
Mar 23 22:17:17.524455 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=103458) setup_logging /opt/stack/neutron/neutron/common/config.py:124}}
Mar 23 22:17:17.524455 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting...
Mar 23 22:17:17.524455 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=103458) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Mar 23 22:17:17.524735 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=103458) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Mar 23 22:17:17.524735 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=103458) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Mar 23 22:17:17.524735 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=103458) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Mar 23 22:17:17.524735 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=103458) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Mar 23 22:17:17.525005 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.manager [-] Loading core plugin: ml2
Mar 23 22:17:17.788915 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.789677 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.789677 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.789677 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.789677 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.789677 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.823517 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.823618 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.823668 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.829392 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.829481 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.829481 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.849744 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.849872 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.849872 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.967949 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.967949 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:17.968138 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.032386 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7966da3371a0> agent after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.032526 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7966da3371a0> agent after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.032591 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _add_segment_host_mapping_for_segment at 0x7966da337240> segment precommit_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.032649 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _delete_segments_for_network at 0x7966da3372e0> network precommit_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.032979 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _prevent_segment_delete_with_port_bound at 0x7966da360540> segment before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091132 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._before_network_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> network before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091285 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function DVRDbMixin._delete_mac_associated_with_agent at 0x7966da325620> agent before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091285 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> network before_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091431 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> port before_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091431 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> port before_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091579 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> segment precommit_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091579 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> segment precommit_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091579 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> segment after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091783 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> segment after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091783 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> network after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091783 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> network precommit_delete 0, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.091783 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> port provisioning_complete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092010 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_create of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> rbac-policy before_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092010 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_delete of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> rbac-policy after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092010 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._retry_binding_revived_agents of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> agent after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092010 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> subnet after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092261 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> subnet precommit_delete 0, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092261 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_tenant of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> rbac-policy before_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092261 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_tenant of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> rbac-policy before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092261 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> segment_host_mapping after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092662 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin.delete_agent_notified of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> agent after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092662 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> rbac-policy before_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092662 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> rbac-policy before_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092662 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7966da639160>> rbac-policy before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.092866 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=103458) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 23 22:17:18.092933 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=103458) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 23 22:17:18.092933 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=103458) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 23 22:17:18.093053 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=103458) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 23 22:17:18.093053 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=103458) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 23 22:17:18.093154 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=103458) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 23 22:17:18.093901 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve']
Mar 23 22:17:18.093901 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public']
Mar 23 22:17:18.094498 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
Mar 23 22:17:18.095041 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]})
Mar 23 22:17:18.095287 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan']
Mar 23 22:17:18.095287 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan'])
Mar 23 22:17:18.096870 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve']
Mar 23 22:17:18.096870 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos']
Mar 23 22:17:18.096870 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos']
Mar 23 22:17:18.096870 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos']
Mar 23 22:17:18.096870 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn']
Mar 23 22:17:18.131972 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn']
Mar 23 22:17:18.132174 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn']
Mar 23 22:17:18.132174 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling.
Mar 23 22:17:18.132722 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7966da069280>> floatingip before_response 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.132866 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7966da069280>> port before_response 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.155963 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat'
Mar 23 22:17:18.155963 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete
Mar 23 22:17:18.155963 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve'
Mar 23 22:17:18.155963 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)]
Mar 23 22:17:18.200130 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO dbcounter [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Registered counter for database neutron
Mar 23 22:17:18.215330 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG oslo_db.sqlalchemy.engines [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103458) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}}
Mar 23 22:17:18.604823 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG dbcounter [-] [103458] Writer thread running {{(pid=103458) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
Mar 23 22:17:18.788104 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Initializing driver for type 'local'
Mar 23 22:17:18.788104 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Initializing driver for type 'vlan'
Mar 23 22:17:18.896350 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] VlanTypeDriver initialization complete
Mar 23 22:17:18.896350 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Initializing extension driver 'port_security'
Mar 23 22:17:18.896350 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.extensions.port_security [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] PortSecurityExtensionDriver initialization complete
Mar 23 22:17:18.896573 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Initializing extension driver 'qos'
Mar 23 22:17:18.896573 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.plugins.ml2.extensions.qos [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QosExtensionDriver initialization complete {{(pid=103458) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}}
Mar 23 22:17:18.896573 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Initializing mechanism driver 'ovn'
Mar 23 22:17:18.896733 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Starting OVNMechanismDriver
Mar 23 22:17:18.898055 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver.pre_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> process before_spawn 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.898143 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver.post_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> process after_init 55550000, True {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.898322 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._add_segment_host_mapping_for_segment of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> segment after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.898322 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver.create_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> segment after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.898533 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver.delete_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> segment after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.898533 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._validate_allowed_address_pairs of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> allowed_address_pair before_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.898660 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> security_group precommit_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.898660 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._update_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> security_group after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.898761 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> security_group after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.898814 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> security_group precommit_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.898870 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> security_group after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.899089 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._create_sg_rule_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> security_group_rule precommit_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.899089 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> security_group_rule after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.899331 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> security_group_rule before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.899331 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> address_group after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.899331 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> address_group after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.899675 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7966da07e120>> address_group after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.899675 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method DriverBase._register of <neutron.services.qos.drivers.ovn.driver.OVNQosDriver object at 0x7966d9d23560>> qos_plugin after_init 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.899848 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNTrunkDriver.register of <neutron.services.trunk.drivers.ovn.trunk_driver.OVNTrunkDriver object at 0x7966d9f2f170>> trunk_plugin after_init 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.899904 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method DriverBase._register of <neutron.services.logapi.drivers.ovn.driver.OVNDriver object at 0x7966d8a1cd40>> logging-plugin after_init 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.900382 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.services.logapi.drivers.ovn.driver [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] OVN logging driver registered
Mar 23 22:17:18.901547 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.plugins.ml2.plugin [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=103458) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
Mar 23 22:17:18.901757 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cf80>> port after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.901871 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cf80>> port after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.901937 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cf80>> port after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.901993 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cf20>> subnet after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902151 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cf20>> subnet after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902151 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cf20>> subnet after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902422 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cec0>> network after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902422 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cec0>> network after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902422 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cec0>> network after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902688 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1ce90>> security_group after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902688 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1ce90>> security_group after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902688 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1ce90>> security_group after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902688 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cef0>> security_group_rule after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902888 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cef0>> security_group_rule after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902888 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1cef0>> security_group_rule after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.902888 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1ce00>> address_group after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.903052 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1ce00>> address_group after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.903052 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7966d8a1ce00>> address_group after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.903160 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ML2 OVO RPC backend initialized. {{(pid=103458) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}}
Mar 23 22:17:18.903834 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.quota [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x7966d8a1e5a0>.
Mar 23 22:17:18.903916 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.plugin [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Modular L2 Plugin initialization complete
Mar 23 22:17:18.903987 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading services supported by the core plugin {{(pid=103458) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}}
Mar 23 22:17:18.904126 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.plugins.ml2.managers [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Got port-security extension from driver 'port_security'
Mar 23 22:17:18.904354 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=103458) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}}
Mar 23 22:17:18.904476 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading Plugin: ovn-router
Mar 23 22:17:18.911416 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._delete_router_remove_external_gateways of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7966d8a1e630>> router before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.911508 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <function L3RpcNotifierMixin._notify_gateway_port_ip_changed at 0x7966da197e20> port after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.911602 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <function L3RpcNotifierMixin._notify_routers_callback at 0x7966da197ce0> port after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.911686 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <function L3RpcNotifierMixin._notify_subnet_gateway_ip_update at 0x7966da197d80> subnet after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.911770 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <function L3RpcNotifierMixin._notify_subnetpool_address_scope_update at 0x7966da197ec0> subnetpool_address_scope after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.911845 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7966d971a7a0> port before_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.911915 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7966d971a7a0> port after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.911995 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method L3_NAT_dbonly_mixin._precommit_delete_port_callback of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7966d8a1e630>> port precommit_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.912079 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_internal_ip_change_for_fip at 0x7966da17fd80> port before_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.912159 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback at 0x7966da17fce0> port before_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.912244 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method RouterAvailabilityZoneMixin._process_az_request of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7966d8a1e630>> router precommit_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.912337 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_create of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7966d8a1e630>> router precommit_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.912416 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_update of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7966d8a1e630>> router precommit_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.912486 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <function OVNL3RouterPlugin._subnet_update at 0x7966d971a700> subnet after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.913079 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.quota.resource_registry [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Tracking information for resource: router configured {{(pid=103458) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 23 22:17:18.913164 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.quota.resource_registry [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Tracking information for resource: floatingip configured {{(pid=103458) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 23 22:17:18.913249 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.services.ovn_l3.plugin [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Starting OVNL3RouterPlugin
Mar 23 22:17:18.913422 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNPortForwarding.register of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7966d8a1e570>> port_forwarding_plugin after_init 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.913587 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method DriverController._check_router_request of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7966d8a1fd70>> router before_create 55550100, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.913675 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method DriverController._clear_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7966d8a1fd70>> router precommit_delete 55550100, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.913745 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method DriverController._set_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7966d8a1fd70>> router precommit_create 55550100, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.913823 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method DriverController._update_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7966d8a1fd70>> router precommit_update 55550100, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.916193 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.services.provider_configuration [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Service providers = [] {{(pid=103458) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:171}}
Mar 23 22:17:18.916287 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.db.servicetype_db [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=103458) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:42}}
Mar 23 22:17:18.917511 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVN_L3_HA_db_mixin._precommit_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> router precommit_create 55549900, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.917730 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_add_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> router_interface after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.917730 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_floatingip_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> floatingip after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.917730 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> floatingip precommit_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.917899 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> floatingip precommit_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.917899 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_floatingip_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> floatingip after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.918008 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_floatingip_status_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> floatingip after_status_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.918067 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_floatingip_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> floatingip after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.918122 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_portforwarding_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> port_forwarding after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.918184 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_remove_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> router_interface after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.918405 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> router after_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.918405 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_router_create_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> router precommit_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.918405 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_router_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> router after_delete 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.918862 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OvnDriver._process_router_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7966d96e2de0>> router after_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.918862 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.services.service_base [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=103458) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}}
Mar 23 22:17:18.918862 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <bound method OVNL3RouterPlugin._post_fork_initialize of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7966d8a1e630>> process after_init 55550001, True {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.918862 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=103458) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 23 22:17:18.918862 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading Plugin: auto_allocate
Mar 23 22:17:18.919647 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7966d9764ae0> network precommit_update 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.919771 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron_lib.callbacks.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7966d9764ae0> network precommit_create 55550000, False {{(pid=103458) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:18.919878 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=103458) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 23 22:17:18.919984 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading Plugin: tag
Mar 23 22:17:18.921981 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=103458) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 23 22:17:18.922119 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading Plugin: timestamp
Mar 23 22:17:18.923010 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=103458) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 23 22:17:18.923093 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading Plugin: network_ip_availability
Mar 23 22:17:18.924261 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=103458) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 23 22:17:18.924313 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading Plugin: flavors
Mar 23 22:17:18.925047 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=103458) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 23 22:17:18.925098 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading Plugin: revisions
Mar 23 22:17:18.925905 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.manager [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=103458) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 23 22:17:18.926014 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG oslo_concurrency.lockutils [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.401s {{(pid=103458) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Mar 23 22:17:18.926177 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=103458) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}}
Mar 23 22:17:18.926254 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Initializing extension manager.
Mar 23 22:17:18.926490 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: __init__.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.926546 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: __pycache__ {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.926642 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: address_group.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.926899 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.927096 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: address-group
Mar 23 22:17:18.927172 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: address_scope.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.927460 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.927601 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: address-scope
Mar 23 22:17:18.927681 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: admin_state_down_before_update.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.927862 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.928005 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins
Mar 23 22:17:18.928087 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: agent.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.928322 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.928465 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: agent
Mar 23 22:17:18.928522 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: agent_resources_synced.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.928715 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.928853 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension agent-resources-synced not supported by any of loaded plugins
Mar 23 22:17:18.928933 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: allowedaddresspairs.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.929465 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.929614 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: allowed-address-pairs
Mar 23 22:17:18.929701 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: auto_allocated_topology.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.929882 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.930025 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: auto-allocated-topology
Mar 23 22:17:18.930115 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: availability_zone.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.930333 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.930462 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: availability_zone
Mar 23 22:17:18.930542 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: availability_zone_filter.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.930721 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.930851 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension availability_zone_filter not supported by any of loaded plugins
Mar 23 22:17:18.930924 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: data_plane_status.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.931103 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.931255 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension data-plane-status not supported by any of loaded plugins
Mar 23 22:17:18.931384 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: default_subnetpools.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.931522 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.931662 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: default-subnetpools
Mar 23 22:17:18.931747 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: dhcpagentscheduler.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.932030 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.932160 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: dhcp_agent_scheduler
Mar 23 22:17:18.932267 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: dns.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.932433 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.932569 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: dns-integration
Mar 23 22:17:18.932645 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: dns_domain_ports.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.932817 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.932955 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: dns-domain-ports
Mar 23 22:17:18.933038 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.933205 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like <project_id>, <project_name>, <user_id> and <user_name> as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.933359 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: dns-integration-domain-keywords
Mar 23 22:17:18.933441 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: dvr.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.933632 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.933766 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension dvr not supported by any of loaded plugins
Mar 23 22:17:18.933843 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: ecmp_routes.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.934010 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.934143 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension ecmp_routes not supported by any of loaded plugins
Mar 23 22:17:18.934245 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: empty_string_filtering.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.934407 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.934540 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: empty-string-filtering
Mar 23 22:17:18.934619 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.934784 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.934918 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins
Mar 23 22:17:18.934993 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.935163 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.935315 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins
Mar 23 22:17:18.935422 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: external_net.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.935601 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.935734 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: external-net
Mar 23 22:17:18.935807 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: extra_dhcp_opt.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.935980 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.936116 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: extra_dhcp_opt
Mar 23 22:17:18.936192 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: extraroute.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.936379 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.936514 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: extraroute
Mar 23 22:17:18.936591 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: extraroute_atomic.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.936768 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.936901 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: extraroute-atomic
Mar 23 22:17:18.936977 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: filter_validation.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.937158 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.937308 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: filter-validation
Mar 23 22:17:18.937364 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: fip_pf_description.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.937547 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.937678 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins
Mar 23 22:17:18.937762 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: fip_pf_detail.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.937922 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.938057 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: floating-ip-port-forwarding-detail
Mar 23 22:17:18.938139 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: fip_pf_port_range.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.938320 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.938454 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins
Mar 23 22:17:18.938624 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: fip_port_details.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.938796 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.938934 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: fip-port-details
Mar 23 22:17:18.939014 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: flavors.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.939204 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.939392 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: flavors
Mar 23 22:17:18.939470 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.939715 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.939853 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins
Mar 23 22:17:18.939933 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: floatingip_pools.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.940130 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.940293 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: floatingip-pools
Mar 23 22:17:18.940385 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: ip_allocation.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.940541 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.940675 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension ip_allocation not supported by any of loaded plugins
Mar 23 22:17:18.940759 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: ip_substring_port_filtering.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.940921 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.941053 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension ip-substring-filtering not supported by any of loaded plugins
Mar 23 22:17:18.941138 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l2_adjacency.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.941316 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.941450 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension l2_adjacency not supported by any of loaded plugins
Mar 23 22:17:18.941539 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.941838 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.941971 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: router
Mar 23 22:17:18.942052 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3_conntrack_helper.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.942528 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.942671 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension l3-conntrack-helper not supported by any of loaded plugins
Mar 23 22:17:18.942735 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.942930 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.943070 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: enable-default-route-bfd
Mar 23 22:17:18.943142 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.943348 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.943503 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: enable-default-route-ecmp
Mar 23 22:17:18.943574 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3_ext_gw_mode.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.943759 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.943889 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: ext-gw-mode
Mar 23 22:17:18.943957 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3_ext_ha_mode.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.944130 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.944283 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: l3-ha
Mar 23 22:17:18.944351 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.944526 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.944660 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins
Mar 23 22:17:18.944730 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3_extra_gws.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.944904 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.945040 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: external-gateway-multihoming
Mar 23 22:17:18.945108 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3_flavors.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.945303 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.945435 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: l3-flavors
Mar 23 22:17:18.945505 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3_ndp_proxy.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.945732 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.945864 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension l3-ndp-proxy not supported by any of loaded plugins
Mar 23 22:17:18.945934 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.946113 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.946262 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins
Mar 23 22:17:18.946333 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: l3agentscheduler.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.946744 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.946881 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension l3_agent_scheduler not supported by any of loaded plugins
Mar 23 22:17:18.946952 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: local_ip.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.947198 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.947415 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension local_ip not supported by any of loaded plugins
Mar 23 22:17:18.947490 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: logging.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.947726 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.947853 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension logging not supported by any of loaded plugins
Mar 23 22:17:18.947923 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: metering.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.948303 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.948525 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension metering not supported by any of loaded plugins
Mar 23 22:17:18.948640 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.948794 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.948932 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins
Mar 23 22:17:18.949001 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: multiprovidernet.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.949184 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.949332 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: multi-provider
Mar 23 22:17:18.949410 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: netmtu.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.949588 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.949714 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: net-mtu
Mar 23 22:17:18.949785 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: netmtu_writable.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.949959 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.950092 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: net-mtu-writable
Mar 23 22:17:18.950161 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: network_availability_zone.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.950376 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.950511 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: network_availability_zone
Mar 23 22:17:18.950581 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: network_ha.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.950757 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.950887 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension network_ha not supported by any of loaded plugins
Mar 23 22:17:18.950965 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: network_ip_availability.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.951148 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.951299 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: network-ip-availability
Mar 23 22:17:18.951394 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: network_ip_availability_details.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.951647 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.951780 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: network-ip-availability-details
Mar 23 22:17:18.951851 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: network_segment_range.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.952097 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.952260 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension network-segment-range not supported by any of loaded plugins
Mar 23 22:17:18.952315 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: pagination.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.952509 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.952651 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: pagination
Mar 23 22:17:18.952739 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: port_device_profile.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.952900 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.953042 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: port-device-profile
Mar 23 22:17:18.953120 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: port_hardware_offload_type.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.953302 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.953439 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension port-hardware-offload-type not supported by any of loaded plugins
Mar 23 22:17:18.953523 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.953682 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.953815 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins
Mar 23 22:17:18.953897 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: port_hints.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.954057 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.954190 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension port-hints not supported by any of loaded plugins
Mar 23 22:17:18.954286 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: port_mac_address_override.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.954456 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.954587 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension port-mac-override not supported by any of loaded plugins
Mar 23 22:17:18.954659 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: port_mac_address_regenerate.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.954829 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.954962 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: port-mac-address-regenerate
Mar 23 22:17:18.955037 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: port_numa_affinity_policy.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.955202 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.955368 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: port-numa-affinity-policy
Mar 23 22:17:18.955424 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.955607 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.955737 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: port-numa-affinity-policy-socket
Mar 23 22:17:18.955813 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: port_resource_request.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.955976 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.956106 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension port-resource-request not supported by any of loaded plugins
Mar 23 22:17:18.956181 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: port_resource_request_groups.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.956374 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.956510 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension port-resource-request-groups not supported by any of loaded plugins
Mar 23 22:17:18.956594 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: port_trusted_vif.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.956751 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.956882 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension port-trusted-vif not supported by any of loaded plugins
Mar 23 22:17:18.956958 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: portbindings.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.957130 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.957286 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: binding
Mar 23 22:17:18.957363 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: portbindings_extended.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.957559 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.957694 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: binding-extended
Mar 23 22:17:18.957771 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: portsecurity.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.957936 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.958071 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: port-security
Mar 23 22:17:18.958150 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: project_id.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.958335 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.958412 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: project-id
Mar 23 22:17:18.958478 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: providernet.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.958649 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.958781 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: provider
Mar 23 22:17:18.958858 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qinq.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.959039 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x8a88 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.959172 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: qinq
Mar 23 22:17:18.959272 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.959965 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.960101 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos not supported by any of loaded plugins
Mar 23 22:17:18.960171 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_bw_limit_direction.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.960399 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.960535 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-bw-limit-direction not supported by any of loaded plugins
Mar 23 22:17:18.960606 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.960787 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.960914 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins
Mar 23 22:17:18.960985 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_default.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.961167 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.961314 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-default not supported by any of loaded plugins
Mar 23 22:17:18.961383 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_fip.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.961559 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.961693 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-fip not supported by any of loaded plugins
Mar 23 22:17:18.961763 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_fip_network_policy.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.961936 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.962068 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-fip-network-policy not supported by any of loaded plugins
Mar 23 22:17:18.962143 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_gateway_ip.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.962330 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.962462 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-gateway-ip not supported by any of loaded plugins
Mar 23 22:17:18.962533 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_port_network_policy.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.962705 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.962837 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-port-network-policy not supported by any of loaded plugins
Mar 23 22:17:18.962908 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.963093 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.963241 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-pps-minimum not supported by any of loaded plugins
Mar 23 22:17:18.963314 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.963511 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.963644 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins
Mar 23 22:17:18.963714 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_pps_rule.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.963905 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.964038 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-pps not supported by any of loaded plugins
Mar 23 22:17:18.964108 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_rule_type_details.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.964298 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.964431 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-rule-type-details not supported by any of loaded plugins
Mar 23 22:17:18.964502 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_rule_type_filter.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.964675 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.964808 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-rule-type-filter not supported by any of loaded plugins
Mar 23 22:17:18.964876 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: qos_rules_alias.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.965061 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.965191 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension qos-rules-alias not supported by any of loaded plugins
Mar 23 22:17:18.965280 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: quota_check_limit.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.965454 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.965586 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: quota-check-limit
Mar 23 22:17:18.965656 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: quota_check_limit_default.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.965848 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.965983 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: quota-check-limit-default
Mar 23 22:17:18.966054 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: quotasv2.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.966550 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.966626 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: quotas
Mar 23 22:17:18.966699 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: quotasv2_detail.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.966957 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.967023 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: quota_details
Mar 23 22:17:18.967095 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: rbac.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.967516 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control tenant access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.967660 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: rbac-policies
Mar 23 22:17:18.967731 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: rbac_address_group.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.967916 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.968062 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension rbac-address-group not supported by any of loaded plugins
Mar 23 22:17:18.968111 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: rbac_address_scope.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.968331 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.968456 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: rbac-address-scope
Mar 23 22:17:18.968535 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: rbac_security_groups.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.968702 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.968836 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: rbac-security-groups
Mar 23 22:17:18.968911 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: rbac_subnetpool.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.969079 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.969206 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension rbac-subnetpool not supported by any of loaded plugins
Mar 23 22:17:18.969302 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: revisionifmatch.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.969488 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.969609 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: revision-if-match
Mar 23 22:17:18.969685 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: revisions.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.969867 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.970001 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: standard-attr-revisions
Mar 23 22:17:18.970091 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: router_availability_zone.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.970294 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.970429 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: router_availability_zone
Mar 23 22:17:18.970504 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: router_enable_snat.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.970676 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.970811 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: router-enable-snat
Mar 23 22:17:18.970887 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: routerservicetype.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.971055 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.971188 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension router-service-type not supported by any of loaded plugins
Mar 23 22:17:18.971293 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: security_groups_default_rules.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.971630 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.971773 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: security-groups-default-rules
Mar 23 22:17:18.971816 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.972017 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.972154 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: security-groups-normalized-cidr
Mar 23 22:17:18.972256 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: security_groups_port_filtering.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.972432 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.972559 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension port-security-groups-filtering not supported by any of loaded plugins
Mar 23 22:17:18.972643 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: security_groups_remote_address_group.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.972804 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.972938 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: security-groups-remote-address-group
Mar 23 22:17:18.973016 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.973226 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.973347 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: security-groups-rules-belongs-to-default-sg
Mar 23 22:17:18.973433 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: security_groups_shared_filtering.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.973593 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.973727 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: security-groups-shared-filtering
Mar 23 22:17:18.973804 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: securitygroup.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.975088 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.975250 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: security-group
Mar 23 22:17:18.975330 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: segment.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.975602 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.975744 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension segment not supported by any of loaded plugins
Mar 23 22:17:18.975795 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.975992 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.976125 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins
Mar 23 22:17:18.976229 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: servicetype.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.976402 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.976540 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: service-type
Mar 23 22:17:18.976595 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: sorting.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.976788 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.976868 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: sorting
Mar 23 22:17:18.976932 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: standard_attr_segment.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.977101 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.977258 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension standard-attr-segment not supported by any of loaded plugins
Mar 23 22:17:18.977343 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: standardattrdescription.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.977527 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.977663 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: standard-attr-description
Mar 23 22:17:18.977746 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: stateful_security_group.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.977909 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.978046 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: stateful-security-group
Mar 23 22:17:18.978121 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: stdattrs_common.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.978292 np0000160382 neutron-ovn-maintenance-worker[103458]: WARNING neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py
Mar 23 22:17:18.978382 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.978541 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.978677 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: subnet-dns-publish-fixed-ip
Mar 23 22:17:18.978761 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: subnet_external_network.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.978921 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.979062 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: subnet-external-network
Mar 23 22:17:18.979139 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: subnet_onboard.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.979482 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.979620 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension subnet_onboard not supported by any of loaded plugins
Mar 23 22:17:18.979702 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: subnet_segmentid_writable.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.979873 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.980016 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension subnet-segmentid-writable not supported by any of loaded plugins
Mar 23 22:17:18.980064 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: subnet_service_types.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.980281 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.980412 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: subnet-service-types
Mar 23 22:17:18.980466 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: subnetallocation.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.980680 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.980807 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: subnet_allocation
Mar 23 22:17:18.980885 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.981071 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.981221 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: subnetpool-prefix-ops
Mar 23 22:17:18.981299 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: tag_creation.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.981473 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.981607 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: tag-creation
Mar 23 22:17:18.981682 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.981854 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.981988 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins
Mar 23 22:17:18.982072 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: tagging.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.982622 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.982764 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: standard-attr-tag
Mar 23 22:17:18.982846 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: timestamp.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.983039 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.983183 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: standard-attr-timestamp
Mar 23 22:17:18.983253 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: trunk.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.983493 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.983627 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension trunk not supported by any of loaded plugins
Mar 23 22:17:18.983714 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: trunk_details.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.983875 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.984012 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension trunk-details not supported by any of loaded plugins
Mar 23 22:17:18.984090 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: uplink_status_propagation.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.984289 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.984416 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension uplink-status-propagation not supported by any of loaded plugins
Mar 23 22:17:18.984492 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.984663 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.984797 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins
Mar 23 22:17:18.984876 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loading extension file: vlantransparent.py {{(pid=103458) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 23 22:17:18.985056 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=103458) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 23 22:17:18.985191 np0000160382 neutron-ovn-maintenance-worker[103458]: INFO neutron.api.extensions [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Loaded extension: vlan-transparent
Mar 23 22:17:18.987309 np0000160382 neutron-ovn-maintenance-worker[103458]: WARNING oslo_service.backend._threading.service [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback.
Mar 23 22:17:18.987599 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Full set of CONF: {{(pid=103458) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}}
Mar 23 22:17:18.987665 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ******************************************************************************** {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
Mar 23 22:17:18.987737 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Configuration options gathered from: {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
Mar 23 22:17:18.987787 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
Mar 23 22:17:18.987902 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
Mar 23 22:17:18.987961 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ================================================================================ {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
Mar 23 22:17:18.988077 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] agent_down_time                = 75 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.988160 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] allow_automatic_dhcp_failover  = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.988263 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] allow_bulk                     = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.988427 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.988514 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] api_extensions_path            =  {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.988603 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] api_paste_config               = api-paste.ini {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.988694 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] api_workers                    = 4 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.988782 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] auth_strategy                  = keystone {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.988867 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] backlog                        = 4096 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.988955 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] base_mac                       = fa:16:3e:00:00:00 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.989075 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] bind_host                      = 0.0.0.0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.989154 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] bind_port                      = 9696 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.989258 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] client_socket_timeout          = 900 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.989350 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] config_dir                     = [] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.989412 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.989504 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] config_source                  = [] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.989595 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] control_exchange               = neutron {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.989682 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] core_plugin                    = ml2 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.989769 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] debug                          = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.989856 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] default_availability_zones     = [] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.989998 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.990090 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] dhcp_agent_notification        = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.990175 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] dhcp_agents_per_network        = 1 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.990277 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] dhcp_lease_duration            = 86400 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.990370 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] dhcp_load_type                 = networks {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.990462 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] dns_domain                     = openstacklocal {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.990546 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.990632 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] enable_default_route_bfd       = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.990716 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] enable_default_route_ecmp      = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.990802 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] enable_dvr                     = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.990885 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] enable_new_agents              = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.990969 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] enable_services_on_agents_with_admin_state_down = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.991053 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] enable_snat_by_default         = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.991136 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] enable_traditional_dhcp        = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.991230 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] external_dns_driver            = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.991320 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] filter_validation              = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.991439 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] global_physnet_mtu             = 1430 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.991510 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] graceful_shutdown_timeout      = 60 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.991623 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] host                           = np0000160382 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.991712 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] host_dvr_for_dhcp              = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.991797 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] http_retries                   = 3 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.991885 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] instance_format                = [instance: %(uuid)s]  {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.991972 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.992065 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ipam_driver                    = internal {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.992147 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ipv6_pd_enabled                = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.992250 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] l3_ha                          = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.992340 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.992428 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] l3_ha_network_physical_name    =  {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.992515 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] l3_ha_network_type             =  {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.992603 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] log_color                      = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.992691 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] log_config_append              = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.992779 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.992863 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] log_dir                        = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.992945 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] log_file                       = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993008 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] log_options                    = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993092 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] log_rotate_interval            = 1 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993179 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] log_rotate_interval_type       = days {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993287 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] log_rotation_type              = none {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993350 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993426 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993496 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993585 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993657 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993735 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] max_allowed_address_pair       = 10 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993818 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] max_dns_nameservers            = 5 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993901 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] max_header_line                = 16384 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.993986 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] max_l3_agents_per_router       = 3 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.994070 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] max_logfile_count              = 30 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.994153 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] max_logfile_size_mb            = 200 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.994262 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] max_routes                     = 30 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.994349 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] max_subnet_host_routes         = 20 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.994435 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] my_ip                          = 162.253.55.52 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.994522 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] my_ipv6                        = 2604:e100:1:0:f816:3eff:feed:ca25 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.994608 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] network_auto_schedule          = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.994690 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] network_link_prefix            = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.994777 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.994864 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] notify_nova_on_port_data_changes = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.994950 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] notify_nova_on_port_status_changes = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995037 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] pagination_max_limit           = -1 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995122 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] periodic_fuzzy_delay           = 5 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995206 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] periodic_interval              = 40 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995308 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] publish_errors                 = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995401 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] rate_limit_burst               = 0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995492 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] rate_limit_except_level        = CRITICAL {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995574 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] rate_limit_interval            = 0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995659 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] retry_until_window             = 30 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995744 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] router_distributed             = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995828 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] rpc_resources_processing_step  = 20 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995912 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] rpc_response_max_timeout       = 600 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.995996 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] rpc_response_timeout           = 60 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.996091 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] rpc_state_report_workers       = 0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.996159 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] rpc_workers                    = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.996261 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] send_events_interval           = 2 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.996362 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] service_plugins                = ['ovn-router'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.996457 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] setproctitle                   = on {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.996535 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] shell_completion               = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.996602 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] state_path                     = /opt/stack/data/neutron {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.996698 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] syslog_log_facility            = LOG_USER {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.996790 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] tcp_keepidle                   = 600 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.996857 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] transport_url                  = **** {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.996949 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] use_journal                    = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.997033 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] use_json                       = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.997125 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] use_ssl                        = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.997203 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] use_stderr                     = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.997306 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] use_syslog                     = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.997385 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] vlan_qinq                      = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.997463 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] vlan_transparent               = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.997539 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] wsgi_default_pool_size         = 100 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.997611 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] wsgi_keep_alive                = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.997705 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.997788 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] wsgi_server_debug              = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:18.997884 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_concurrency.disable_process_locking = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.997978 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.998075 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler.connection_string     = messaging:// {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.998164 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler.enabled               = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.998278 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler.es_doc_type           = notification {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.998363 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler.es_scroll_size        = 10000 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.998455 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler.es_scroll_time        = 2m {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.998543 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler.filter_error_trace    = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.998640 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler.hmac_keys             = **** {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.998713 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler.sentinel_service_name = mymaster {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.998810 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler.socket_timeout        = 0.1 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.998906 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler.trace_requests        = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.998989 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler.trace_sqlalchemy      = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.999072 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler_jaeger.process_tags   = {} {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.999158 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler_jaeger.service_name_prefix = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.999256 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] profiler_otlp.service_name_prefix = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.999351 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.999469 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.999569 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.999696 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.999786 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:18.999882 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_policy.enforce_new_defaults = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.000051 np0000160382 neutron-ovn-maintenance-worker[103458]: WARNING oslo_config.cfg [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.).  Its value may be silently ignored in the future.
Mar 23 22:17:19.000146 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_policy.enforce_scope      = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.000259 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_policy.policy_default_rule = default {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.000354 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.000501 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.000538 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.000627 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.000742 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.000831 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.000916 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001007 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_policy.remote_timeout     = 60.0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001099 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001174 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] service_providers.service_provider = [] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001291 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.auth_section              = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001387 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.auth_type                 = password {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001478 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.auth_url                  = https://162.253.55.52/identity {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001623 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.cafile                    = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001670 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.certfile                  = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001748 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.collect_timing            = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001817 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.default_domain_id         = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001893 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.default_domain_name       = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.001981 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.domain_id                 = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.002073 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.domain_name               = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.002152 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.endpoint_type             = public {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.002258 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.insecure                  = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.002340 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.keyfile                   = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.002429 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.password                  = **** {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.002511 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.project_domain_id         = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.002605 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.project_domain_name       = Default {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.002675 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.project_id                = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.002774 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.project_name              = service {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.002858 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.region_name               = RegionOne {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.002948 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.split_loggers             = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.003040 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.system_scope              = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.003117 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.timeout                   = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.003191 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.trust_id                  = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.003303 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.user_domain_id            = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.003415 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.user_domain_name          = Default {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.003487 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.user_id                   = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.003575 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] nova.username                  = nova {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.003666 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] placement.auth_section         = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.003740 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] placement.auth_type            = password {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.003837 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] placement.cafile               = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.003922 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] placement.certfile             = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004007 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] placement.collect_timing       = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004091 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] placement.endpoint_type        = public {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004176 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] placement.insecure             = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004279 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] placement.keyfile              = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004366 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] placement.region_name          = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004451 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] placement.split_loggers        = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004533 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] placement.timeout              = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004622 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.auth_section            = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004707 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.auth_type               = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004790 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.cafile                  = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004872 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.certfile                = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.004957 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.collect_timing          = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005042 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.connect_retries         = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005125 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.connect_retry_delay     = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005224 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.enable_notifications    = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005318 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.endpoint_override       = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005404 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.insecure                = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005488 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.interface               = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005570 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.keyfile                 = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005654 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.max_version             = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005737 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.min_version             = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005822 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.region_name             = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005906 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.retriable_status_codes  = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.005990 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.service_name            = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.006073 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.service_type            = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.006159 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.split_loggers           = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.006262 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.status_code_retries     = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.006348 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.status_code_retry_delay = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.006431 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.timeout                 = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.006516 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.valid_interfaces        = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.006599 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ironic.version                 = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.006689 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] cli_script.dry_run             = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.006785 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.006872 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.006958 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.007044 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.007130 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.007228 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.007322 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.007439 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.007524 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.007615 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.hostname = np0000160382 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.007700 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.007791 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.007880 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.007967 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.008060 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.008149 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.008251 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.008339 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.008471 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.008513 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.008600 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.008680 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.008764 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.008849 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.008933 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009019 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009104 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009189 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009333 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009361 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009454 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.ssl      = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009542 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009631 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009718 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009807 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009895 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.009980 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.010071 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_notifications.driver = [] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.010158 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_notifications.retry = -1 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.010449 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.010449 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_messaging_notifications.transport_url = **** {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.010449 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_reports.file_event_handler = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.010541 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.010614 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_reports.log_dir           = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.010685 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.010796 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QUOTAS.default_quota           = -1 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.010864 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.010978 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QUOTAS.quota_floatingip        = 50 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.011066 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QUOTAS.quota_network           = 100 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.011118 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QUOTAS.quota_port              = 500 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.011990 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QUOTAS.quota_rbac_policy       = 10 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.011990 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QUOTAS.quota_router            = 10 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.011990 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QUOTAS.quota_security_group    = 10 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.011990 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QUOTAS.quota_security_group_rule = 100 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.011990 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QUOTAS.quota_subnet            = 100 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.011990 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] QUOTAS.track_quota_usage       = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.011990 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012166 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep.group                  = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012166 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep.helper_command         = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012166 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep.log_daemon_traceback   = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012166 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep.logger_name            = oslo_privsep.daemon {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012166 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep.thread_pool_size       = 16 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012297 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep.user                   = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012322 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012713 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_dhcp_release.group     = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012713 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_dhcp_release.helper_command = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012713 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012713 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.012713 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_dhcp_release.thread_pool_size = 16 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013081 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_dhcp_release.user      = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013081 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013081 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_ovs_vsctl.group        = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013081 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_ovs_vsctl.helper_command = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013389 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013389 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013389 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013389 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_ovs_vsctl.user         = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013484 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_namespace.capabilities = [21] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013670 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_namespace.group        = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013670 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_namespace.helper_command = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013799 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_namespace.log_daemon_traceback = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013799 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.013925 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_namespace.thread_pool_size = 16 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014002 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_namespace.user         = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014002 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_conntrack.capabilities = [12] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014119 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_conntrack.group        = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014247 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_conntrack.helper_command = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014332 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_conntrack.log_daemon_traceback = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014418 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014451 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_conntrack.thread_pool_size = 16 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014540 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_conntrack.user         = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014632 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_link.capabilities      = [12, 21] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014712 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_link.group             = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014806 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_link.helper_command    = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014897 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_link.log_daemon_traceback = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.014958 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.015057 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_link.thread_pool_size  = 16 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.015117 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] privsep_link.user              = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.015234 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.015328 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.015418 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] AGENT.use_helper_for_ns_read   = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.015519 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] SECURITYGROUP.enable_ipset     = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.015612 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] SECURITYGROUP.enable_security_group = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.015696 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] SECURITYGROUP.firewall_driver  = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.015800 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.015885 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] experimental.ipv6_pd_enabled   = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.015967 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.016048 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2.external_network_type      = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.016141 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2.mechanism_drivers          = ['ovn'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.016251 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2.overlay_ip_version         = 4 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.016340 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2.path_mtu                   = 0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.016427 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2.physical_network_mtus      = {} {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.016521 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2.project_network_types      = ['geneve'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.016612 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.016707 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.016803 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2_type_flat.flat_networks    = ['public'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.016899 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2_type_geneve.max_header_size = 38 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.016992 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.017088 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.017179 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] network_log.burst_limit        = 25 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.017283 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] network_log.local_output_log_base = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.017369 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] network_log.rate_limit         = 100 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.017459 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.asyncio_connection    = **** {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.017541 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.asyncio_slave_connection = **** {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.017630 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.backend               = sqlalchemy {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.017717 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.connection            = **** {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.017804 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.connection_debug      = 0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.017892 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.connection_parameters =  {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.017977 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.connection_recycle_time = 3600 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.018060 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.connection_trace      = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.018146 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.db_inc_retry_interval = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.018250 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.db_max_retries        = 20 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.018337 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.db_max_retry_interval = 10 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.018424 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.db_retry_interval     = 1 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.018509 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.max_overflow          = 50 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.018595 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.max_pool_size         = 5 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.018680 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.max_retries           = 10 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.018768 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.018852 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.mysql_wsrep_sync_wait = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.018936 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.pool_timeout          = None {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.019023 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.retry_interval        = 10 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.019122 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.slave_connection      = **** {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.019203 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.sqlite_synchronous    = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.019309 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.synchronous_reader    = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.019449 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] database.use_db_reconnect      = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.019544 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.bfd_min_rx                 = 1000 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.019570 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.bfd_min_tx                 = 100 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020170 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.bfd_mult                   = 3 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020170 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.broadcast_arps_to_all_routers = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020170 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.dhcp_default_lease_time    = 43200 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020170 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020170 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.dns_records_ovn_owned      = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020170 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.dns_servers                = [] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020170 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.enable_distributed_floating_ip = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020608 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.fdb_age_threshold          = 300 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020608 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ha_failover_strategy       = normal {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020608 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.live_migration_activation_strategy = rarp {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020608 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.localnet_learn_fdb         = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020608 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.mac_binding_age_threshold  = 300 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020896 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.neutron_sync_mode          = log {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020896 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_dhcp4_global_options   = {} {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020896 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_dhcp6_global_options   = {} {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020896 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_l3_scheduler           = leastloaded {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.020979 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_metadata_enabled       = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.021076 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.021143 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.021261 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_nb_connection          = ['ssl:162.253.55.52:6641'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.021351 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.021438 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_router_indirect_snat   = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.021529 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.021615 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.021706 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_sb_connection          = ['ssl:162.253.55.52:6642'] {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.021795 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.021881 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovs_create_tap             = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.021968 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovsdb_connection_timeout   = 180 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.022058 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovsdb_log_level            = INFO {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.022143 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovsdb_probe_interval       = 60000 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.022247 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.ovsdb_retry_max_interval   = 180 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.022334 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.stateless_nat_enabled      = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.022424 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.022513 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] OVS.bridge_mac_table_size      = 50000 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.022598 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] OVS.igmp_flood                 = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.022682 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] OVS.igmp_flood_reports         = True {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.022767 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] OVS.igmp_flood_unregistered    = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.022856 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] OVS.igmp_snooping_enable       = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.022941 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] OVS.ovsdb_timeout              = 10 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.023035 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.023120 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn_nb_global.ignore_lsp_down  = False {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.023231 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.023294 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG cotyledon.oslo_config_glue [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] ******************************************************************************** {{(pid=103458) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
Mar 23 22:17:19.023375 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG oslo_service.backend._threading.service [None req-c0b6ac30-889c-4ef0-9544-9f0c75a841e0 None None] Selected the multiprocessing context: <multiprocessing.context.ForkContext object at 0x7966de73cc80> and updated it in Cotyledon ServiceManager: <multiprocessing.context.ForkContext object at 0x7966de73cc80> {{(pid=103458) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:394}}
Mar 23 22:17:19.052918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=104232) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}}
Mar 23 22:17:19.053481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
Mar 23 22:17:19.053481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
Mar 23 22:17:19.053481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
Mar 23 22:17:19.053481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
Mar 23 22:17:19.053481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
Mar 23 22:17:19.053725 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time                = 75 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.053725 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover  = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.053819 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk                     = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.054078 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.054201 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path            =  {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.054337 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config               = api-paste.ini {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.054439 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] api_workers                    = 4 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.054539 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy                  = keystone {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.054692 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] backlog                        = 4096 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.054803 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] base_mac                       = fa:16:3e:00:00:00 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.055029 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] bind_host                      = 0.0.0.0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.055164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] bind_port                      = 9696 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout          = 900 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.055459 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] config_dir                     = [] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.055580 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.055655 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] config_source                  = [] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.055773 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] control_exchange               = neutron {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.055878 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] core_plugin                    = ml2 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.055976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] debug                          = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.056087 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones     = [] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.056328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.056447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification        = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.056599 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network        = 1 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.056707 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration            = 86400 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.056807 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type                 = networks {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.056906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] dns_domain                     = openstacklocal {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.057003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.057101 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd       = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.057198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp      = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.057349 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr                     = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.057461 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents              = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.057563 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.057663 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default         = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.057723 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp        = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.057781 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver            = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.057936 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations             = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.057936 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] filter_validation              = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.058133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu             = 1430 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.058133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout      = 60 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.058501 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] host                           = np0000160382 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.058644 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp              = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.058699 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] http_retries                   = 3 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.058903 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] instance_format                = [instance: %(uuid)s]  {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.058903 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.059350 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver                    = internal {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.059350 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled                = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.059350 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] l3_ha                          = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.059350 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.059918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name    =  {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.059918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type             =  {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.059918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] log_color                      = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.059918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] log_config_append              = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.059918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.059918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] log_dir                        = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060269 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] log_file                       = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060269 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] log_options                    = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060269 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval            = 1 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060269 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type       = days {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060269 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type              = none {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair       = 10 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.060676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers            = 5 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061028 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] max_header_line                = 16384 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061028 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router       = 3 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061028 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count              = 30 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061028 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb            = 200 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061028 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] max_routes                     = 30 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061028 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes         = 20 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061331 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] my_ip                          = 162.253.55.52 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061331 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6                        = 2604:e100:1:0:f816:3eff:feed:ca25 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061331 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule          = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061331 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix            = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061525 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061525 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061525 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061688 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit           = -1 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061688 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay           = 5 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval              = 40 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] publish_errors                 = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061883 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst               = 0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061940 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level        = CRITICAL {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.061992 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval            = 0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062060 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] retry_until_window             = 30 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062118 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] router_distributed             = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062173 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step  = 20 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062253 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout       = 600 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062351 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout           = 60 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers       = 0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062473 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers                    = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062529 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval           = 2 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062614 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] service_plugins                = ['ovn-router'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062693 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] setproctitle                   = on {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062752 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] shell_completion               = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] state_path                     = /opt/stack/data/neutron {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility            = LOG_USER {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle                   = 600 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.062980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] transport_url                  = **** {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] use_journal                    = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063114 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] use_json                       = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063167 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] use_ssl                        = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063254 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] use_stderr                     = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063330 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] use_syslog                     = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063458 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq                      = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063458 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent               = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size         = 100 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive                = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug              = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 23 22:17:19.063792 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.063950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064048 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string     = messaging:// {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064082 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled               = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064139 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type           = notification {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064208 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size        = 10000 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064292 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time        = 2m {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064363 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace    = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys             = **** {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064521 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout        = 0.1 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064669 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests        = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064739 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy      = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags   = {} {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064885 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.064948 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065036 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065121 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065199 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name =  {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065298 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065362 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065615 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope      = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065656 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065690 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065753 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065851 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065899 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065940 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.065981 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066030 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout     = 60.0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066095 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066150 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section              = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066291 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type                 = password {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066334 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url                  = https://162.253.55.52/identity {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile                    = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile                  = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing            = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id         = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name       = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id                 = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name               = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type             = public {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure                  = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile                   = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.password                  = **** {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id         = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name       = Default {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id                = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.066919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name              = service {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name               = RegionOne {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers             = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope              = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout                   = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id                  = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067162 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id            = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067162 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name          = Default {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id                   = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] nova.username                  = nova {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067326 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section         = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067373 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type            = password {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067423 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile               = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067464 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile             = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067505 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing       = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067546 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type        = public {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067581 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure             = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067624 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile              = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name          = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067702 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers        = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067742 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout              = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067790 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section            = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type               = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067872 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile                  = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067913 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile                = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067948 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing          = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.067992 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries         = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068042 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay     = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068083 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications    = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068124 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override       = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068165 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure                = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068205 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface               = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068280 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile                 = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068322 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version             = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068358 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version             = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068401 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name             = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes  = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068479 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name            = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068519 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type            = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068558 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers           = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068607 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries     = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068712 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout                 = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces        = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068773 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ironic.version                 = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068804 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run             = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068876 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.068975 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069026 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069069 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069109 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069145 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069188 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069249 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069300 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000160382 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069342 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069395 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069441 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069543 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069590 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069637 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069688 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069819 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069861 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069900 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069942 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.069979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070020 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070053 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl      = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file =  {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version =  {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.070880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071179 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir           = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071179 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071179 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota           = -1 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071179 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071179 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip        = 50 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071179 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network           = 100 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071338 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port              = 500 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071338 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy       = 10 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071338 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router            = 10 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group    = 10 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071496 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet            = 100 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage       = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071581 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071629 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep.group                  = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071664 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command         = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback   = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071764 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name            = oslo_privsep.daemon {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071798 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size       = 16 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071855 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep.user                   = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071907 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group     = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.071990 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072030 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072078 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072120 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 16 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072161 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user      = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072208 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group        = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072311 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072360 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072401 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072441 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072478 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user         = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072535 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072577 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group        = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072617 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072657 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072698 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072746 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 16 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072788 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user         = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072834 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group        = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072952 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.072993 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 16 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073073 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user         = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073128 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities      = [12, 21] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073170 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group             = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073227 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command    = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073273 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073310 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size  = 16 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073393 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user              = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073460 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073505 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read   = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073619 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset     = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073721 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver  = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073768 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073830 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled   = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073891 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073933 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type      = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.073976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers          = ['ovn'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074018 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version         = 4 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu                   = 0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074102 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus      = {} {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types      = ['geneve'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074305 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks    = ['public'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074389 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit        = 25 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074561 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074597 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit         = 100 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection    = **** {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074706 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074752 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.backend               = sqlalchemy {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074799 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.connection            = **** {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074847 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug      = 0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074890 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters =  {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074941 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.074975 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace      = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075061 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries        = 20 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075102 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075143 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval     = 1 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075184 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow          = 50 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075241 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size         = 5 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075280 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries           = 10 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075332 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode        = TRADITIONAL {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075366 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075442 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout          = None {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075499 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval        = 10 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075549 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection      = **** {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075590 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous    = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075636 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader    = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075677 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect      = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx                 = 1000 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075787 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx                 = 100 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075840 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult                   = 3 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075875 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075922 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time    = 43200 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075954 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.075999 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned      = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076039 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers                = [] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076080 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold          = 300 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076160 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy       = normal {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076201 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076261 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb         = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076303 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold  = 300 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076353 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode          = log {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076395 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options   = {} {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076432 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options   = {} {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler           = leastloaded {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076511 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled       = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076563 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076616 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076658 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection          = ['ssl:162.253.55.52:6641'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076699 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076740 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat   = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076780 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076830 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076872 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection          = ['ssl:162.253.55.52:6642'] {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap             = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.076979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout   = 180 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level            = INFO {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval       = 60000 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077112 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval   = 180 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077144 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled      = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077190 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077259 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size      = 50000 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077310 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood                 = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077358 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports         = True {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077400 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered    = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077449 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable       = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077490 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout              = 10 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077588 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down  = False {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077636 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 23 22:17:19.077677 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=104232) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
Mar 23 22:17:19.077835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [104232] {{(pid=104232) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}}
Mar 23 22:17:19.079233 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-4027568', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-1902539'] for process (None), after_init {{(pid=104232) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Mar 23 22:17:19.081472 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry
Mar 23 22:17:19.131205 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Mar 23 22:17:19.131369 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Mar 23 22:17:19.131369 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Mar 23 22:17:19.131369 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.131562 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.131562 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.131562 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.131727 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.131727 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.131727 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.131727 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.131923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.131923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.131923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.131923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.134676 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:162.253.55.52:6641: connecting...
Mar 23 22:17:19.140978 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:162.253.55.52:6641: connected
Mar 23 22:17:19.172552 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry
Mar 23 22:17:19.198869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Mar 23 22:17:19.198869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Mar 23 22:17:19.198869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.198869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=104232) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 23 22:17:19.201240 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:162.253.55.52:6642: connecting...
Mar 23 22:17:19.209002 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:162.253.55.52:6642: connected
Mar 23 22:17:19.244117 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7966d8a5f560>> floatingip precommit_update 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.244117 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7966d8a5f560>> floatingip precommit_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.244480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7966d8a5f560>> floatingip before_create 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.244480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7966d8a5f560>> floatingip before_update 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.244480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7966d8a5f560>> port after_update 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.244480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7966d8a5f560>> port precommit_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.244743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-1902439'] for port_forwarding_plugin (None), after_init {{(pid=104232) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Mar 23 22:17:19.244743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7966d8a1e570>> port_forwarding after_create 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.244909 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7966d8a1e570>> router_interface after_create 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.244909 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7966d8a1e570>> port_forwarding after_update 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.244909 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7966d8a1e570>> port_forwarding after_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.245154 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7966d8a1e570>> router_interface after_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.245154 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Mar 23 22:17:19.245392 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension
Mar 23 22:17:19.245625 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._prevent_segment_delete_with_subnet_associated of <neutron.services.segments.plugin.Plugin object at 0x7966d8a5f500>> segment before_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.245749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._validate_auto_address_subnet_delete of <neutron.services.segments.plugin.Plugin object at 0x7966d8a5f500>> subnet precommit_delete_associations 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.245969 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_addition_to_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7966d8a5e510>> segment_host_mapping after_create 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.246097 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_removal_from_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7966d8a5e510>> segment_host_mapping after_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.246097 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7966d8a5e510>> port after_create 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.246302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7966d8a5e510>> port after_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.246302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7966d8a5e510>> port after_update 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.246302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_segment_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7966d8a5e510>> segment after_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.246302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_created of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7966d8a5e510>> subnet after_create 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.246620 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7966d8a5e510>> subnet after_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.246620 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7966d8a5e510>> subnet after_update 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.307287 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7966d968a2a0>> subnet after_create 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.307447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_delete of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7966d968a2a0>> subnet after_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.307447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7966d968a2a0>> subnet before_create 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.307447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_update of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7966d968a2a0>> subnet before_update 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.309634 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_resource_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7966d8a5e3c0>> security_group after_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.309729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_target_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7966d8a5e3c0>> port after_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.309830 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-1513554'] for logging-plugin (None), after_init {{(pid=104232) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Mar 23 22:17:19.309953 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7966d8ab5040>> security_group_rule after_create 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.310001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7966d8ab5040>> security_group_rule after_update 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:19.310063 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7966d8ab5040>> security_group_rule after_delete 55550000, False {{(pid=104232) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 23 22:17:28.792646 np0000160382 neutron-ovn-maintenance-worker[103458]: DEBUG dbcounter [-] [103458] Writing DB stats neutron:SELECT=4 {{(pid=103458) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:17:29.311447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-03-23 22:17:29.310953 {{(pid=104232) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}}
Mar 23 22:17:29.314129 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync port groups started @ 2026-03-23 22:17:29.313672 {{(pid=104232) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}}
Mar 23 22:17:29.330169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writer thread running {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
Mar 23 22:17:29.343123 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync port groups completed @ 2026-03-23 22:17:29.342709 {{(pid=104232) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}}
Mar 23 22:17:29.343303 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-03-23 22:17:29.343069 {{(pid=104232) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1125}}
Mar 23 22:17:29.450036 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.services.segments.db [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (<neutron_lib.context.Context object at 0x7966d8a1e000>,) {'filters': {'network_id': ['06cffee1-b8de-433e-a67a-ec0e950c9238'], 'is_dynamic': False}} {{(pid=104232) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
Mar 23 22:17:29.454981 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync metadata ports started {{(pid=104232) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1070}}
Mar 23 22:17:29.516241 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync metadata ports completed {{(pid=104232) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1122}}
Mar 23 22:17:29.516241 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=104232) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:922}}
Mar 23 22:17:29.747849 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-03-23 22:17:29.747551 {{(pid=104232) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:995}}
Mar 23 22:17:29.748509 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Mar 23 22:17:29.748822 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=104232) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1000}}
Mar 23 22:17:29.748949 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=104232) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1059}}
Mar 23 22:17:29.749044 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-03-23 22:17:29.748922 {{(pid=104232) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1294}}
Mar 23 22:17:29.749264 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync ACLs started @ 2026-03-23 22:17:29.749063 {{(pid=104232) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}}
Mar 23 22:17:29.782995 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync ACLs completed @ 2026-03-23 22:17:29.782728 {{(pid=104232) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:429}}
Mar 23 22:17:29.783150 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync routers and router ports started @ 2026-03-23 22:17:29.782997 {{(pid=104232) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:596}}
Mar 23 22:17:29.829693 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Mar 23 22:17:29.829984 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync routers and router ports completed @ 2026-03-23 22:17:29.829747 {{(pid=104232) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:917}}
Mar 23 22:17:29.830072 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync port QoS policies started @ 2026-03-23 22:17:29.829972 {{(pid=104232) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1342}}
Mar 23 22:17:29.830176 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] Starting OVNClientQosExtension
Mar 23 22:17:29.845800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync port QoS policies completed @ 2026-03-23 22:17:29.845590 {{(pid=104232) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1365}}
Mar 23 22:17:29.845982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync Floating IP QoS policies started @ 2026-03-23 22:17:29.845772 {{(pid=104232) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1370}}
Mar 23 22:17:29.845982 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] Starting OVNClientQosExtension
Mar 23 22:17:29.867766 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-03-23 22:17:29.867612 {{(pid=104232) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1393}}
Mar 23 22:17:29.867862 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync Floating IP NAT rules started @ 2026-03-23 22:17:29.867749 {{(pid=104232) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1398}}
Mar 23 22:17:29.868058 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-03-23 22:17:29.867946 {{(pid=104232) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1421}}
Mar 23 22:17:29.868141 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8d1a336e-9f16-47e1-8bb2-5e4f4b2ca68c None None] OVN-Northbound DB sync process completed @ 2026-03-23 22:17:29.868042 {{(pid=104232) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}}
Mar 23 22:17:39.312929 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-03-23 22:17:39.312421 {{(pid=104232) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1449}}
Mar 23 22:17:39.313908 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Mar 23 22:17:39.315019 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-b6658ed5-b702-462c-a5f1-55a0cf7ae88c None None] OVN-SB Sync hostname and physical networks started @ 2026-03-23 22:17:39.314502 {{(pid=104232) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1476}}
Mar 23 22:17:39.317743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.317831 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.317906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.317962 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.318430 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.318511 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.318585 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.318669 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.318743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.318821 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.318896 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.318968 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.319058 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.319115 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.319202 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.319312 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.319388 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.319465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.319538 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.319610 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.319689 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_qos_fip_rule_priority {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.319790 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.320034 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_static_routes {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.320502 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=104232) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Mar 23 22:17:39.322449 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.323020 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started
Mar 23 22:17:39.323425 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization
Mar 23 22:17:39.324492 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.328583 np0000160382 neutron-ovn-maintenance-worker[104232]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-b6658ed5-b702-462c-a5f1-55a0cf7ae88c None None] New host np0000160382 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron
Mar 23 22:17:39.329765 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.331155 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.340907 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.343906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.347918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.349814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.353257 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.355449 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.357971 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.367523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b11314c0-2f03-4b78-b3b7-34f3839adb24 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:39.370087 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.370741 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Mar 23 22:17:39.371198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.378578 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG oslo_concurrency.lockutils [None req-b6658ed5-b702-462c-a5f1-55a0cf7ae88c None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=104232) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Mar 23 22:17:39.380406 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.382155 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG oslo_concurrency.lockutils [None req-b6658ed5-b702-462c-a5f1-55a0cf7ae88c None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.002s {{(pid=104232) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Mar 23 22:17:39.383338 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.389480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.406538 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.411604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.413909 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:39.414186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.431972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:39.431972 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.431972 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:39.431972 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.431972 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.431972 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:39.431972 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.431972 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.521920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.522522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:39.540406 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.544687 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:39.551018 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron_lib.callbacks.manager [None req-b6658ed5-b702-462c-a5f1-55a0cf7ae88c None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-152580', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-2752935'] for segment_host_mapping (None), after_create {{(pid=104232) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Mar 23 22:17:39.556415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Maintenance task: No inconsistencies found. Skipping {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}}
Mar 23 22:17:39.557181 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.db.ovn_hash_ring_db [None req-36d8e3b7-7119-4003-94e9-f96697fa81c3 None None] Cleaned up Hash Ring nodes older than 5 days
Mar 23 22:17:39.566342 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.567678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:17:39.571919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:39.580357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.587948 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.589538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:39.589538 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.589538 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:39.589538 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.589538 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.589538 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:39.589538 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.589538 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:39.591944 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:17:39.639568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:39.640769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:17:39.640769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.640769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:17:39.640769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.640769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.640769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:17:39.640769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.640769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:17:39.643906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.647248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 919, in configure_nb_global
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:17:39.674431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.675341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:39.676245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.676245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:39.676245 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.681286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:39.681286 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.681286 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:39.681286 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.681286 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.681286 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:39.681286 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.681286 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:39.690310 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:39.692815 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [None req-9d41a5c6-8cd0-49da-a3b4-e79b7a5348f2 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.693654 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.services.segments.db [None req-b6658ed5-b702-462c-a5f1-55a0cf7ae88c None None] Segments {'a8bd993e-d23e-4e73-8066-766307763bdb'} mapped to the host np0000160382 {{(pid=104232) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}}
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:17:39.698228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:39.698700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:17:39.698700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.698700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:17:39.698700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.698700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.698700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:17:39.698700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.698700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:17:39.705777 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.710259 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Maintenance task: Check missing prefix router_name in external_ids of LRPs. {{(pid=104232) update_lrouter_ports_ext_ids_name_prefix /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:503}}
Mar 23 22:17:39.713478 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.714015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [None req-7a9907c9-992b-4cd8-8269-513ef47484ca None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.714715 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:17:39.726413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.727526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.728755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:39.728755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.728755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:39.728755 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG oslo_concurrency.lockutils [None req-b6658ed5-b702-462c-a5f1-55a0cf7ae88c None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.326s {{(pid=104232) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Mar 23 22:17:39.759308 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:39.763126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.775715 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-b6658ed5-b702-462c-a5f1-55a0cf7ae88c None None] OVN-SB Sync hostname and physical networks completed @ 2026-03-23 22:17:39.727767 {{(pid=104232) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1503}}
Mar 23 22:17:39.781408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_qos_fip_rule_priority' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.789883 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Maintenance task: Check missing prefix router_name in LRPs (took 0.06 seconds)
Mar 23 22:17:39.793961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:39.793961 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.793961 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:39.793961 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.793961 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.793961 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:39.793961 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.793961 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:39.796982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.797207 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:39.804845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.813002 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_qos_fip_rule_priority' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:17:39.835564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:39.836024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:17:39.836024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.836024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:17:39.836024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.836024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.836024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:17:39.836024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.836024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:17:39.840581 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:39.849578 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:17:39.871961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.872507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:39.873006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.873006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:39.873006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.883663 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.887673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:39.887673 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.887673 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:39.887673 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.887673 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.887673 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:39.887673 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.887673 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.890872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.891807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:39.892919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=104232) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 23 22:17:39.900064 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [None req-ed247e61-87db-4aef-baaf-03c463a43e4a None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.930518 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [None req-36d8e3b7-7119-4003-94e9-f96697fa81c3 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:39.975371 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:39.975824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.979178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b11314c0-2f03-4b78-b3b7-34f3839adb24 None None] Traceback (most recent call last):
Mar 23 22:17:39.979178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.979178 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:39.979178 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.979178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.979178 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:39.979178 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.979178 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-b11314c0-2f03-4b78-b3b7-34f3839adb24 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.982742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:39.983610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:39.985466 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [None req-53348232-01be-45f9-9335-c6f024123f71 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:40.020011 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [None req-bcc9c793-fee4-4e04-b95e-f604c0d11974 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:40.102889 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:40.117143 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=104232) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 23 22:17:40.156765 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=bf652fbc-ad1a-4f7a-a6ac-b7b30281e79f, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-06cffee1-b8de-433e-a67a-ec0e950c9238', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-30981653-9cf2-456c-b4c8-6836a78525ee', 'neutron:subnet_ids': 'fb1912da-e258-4d41-8cd4-7896f0de6385'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={'address_mode': 'slaac', 'mtu': '1372', 'send_periodic': 'true'}, mac='fa:16:3e:b4:ae:1c', name='lrp-a8499535-866f-4bad-b671-ec46d6e7a76d', networks=['fdc3:b1c9:e405::1/64'], options={}, peer=[], status={}) old= {{(pid=104232) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Mar 23 22:17:40.158417 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Mar 23 22:17:40.245275 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-06cffee1-b8de-433e-a67a-ec0e950c9238 synchronized; highest priority chassis 3871dd15-d123-44a6-815e-51587164f440
Mar 23 22:17:40.246508 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '06cffee1-b8de-433e-a67a-ec0e950c9238', 'neutron:router_id': '30981653-9cf2-456c-b4c8-6836a78525ee'}),), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:40.247195 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, chassis=3871dd15-d123-44a6-815e-51587164f440, priority=1, columns={}) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:40.247539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:40.251168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:40.251168 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:40.251168 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:40.251168 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:40.251168 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:40.251168 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:40.251168 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:40.251168 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event Traceback (most recent call last):
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     match.run(event, row, updates)
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 85, in run
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2385, in link_network_ha_chassis_group
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     next(self.gen)
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     next(self.gen)
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with self.create_transaction(
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     self.result = self.commit()
Mar 23 22:17:40.254659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Mar 23 22:17:40.255734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:40.255734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     raise result.ex
Mar 23 22:17:40.255734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:40.255734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Mar 23 22:17:40.255734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Mar 23 22:17:40.255734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:40.255734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Mar 23 22:17:40.255734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:40.255734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event 
Mar 23 22:17:40.337408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Mar 23 22:17:40.337998 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-b6658ed5-b702-462c-a5f1-55a0cf7ae88c None None] OVN-Southbound DB sync process completed @ 2026-03-23 22:17:40.337542 {{(pid=104232) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1457}}
Mar 23 22:17:41.934475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=68679f94-7562-4b6b-9f37-7c9d6c9de5cb, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-bb9d5c47-4694-42c2-a018-18e3c9058915', 'neutron:revision_number': '8', 'neutron:router_name': 'neutron-30981653-9cf2-456c-b4c8-6836a78525ee', 'neutron:subnet_ids': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a a53179c9-417a-4560-9bc7-e3f64551b15b'}, gateway_chassis=[49ac8d3e-10e6-4b8d-aac7-b56ea0d9a22a], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:92:fd:20', name='lrp-27af92ce-bee7-4ee0-bb53-804cca17b9c9', networks=['172.24.5.147/24', '2001:db8::234/64'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=104232) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Mar 23 22:17:41.935122 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=d92c1885-d436-44cb-9436-35c29d897dc2, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-bb9d5c47-4694-42c2-a018-18e3c9058915', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-30981653-9cf2-456c-b4c8-6836a78525ee', 'neutron:subnet_ids': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a'}, gateway_chassis=[49ac8d3e-10e6-4b8d-aac7-b56ea0d9a22a], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:92:fd:20', name='lrp-27af92ce-bee7-4ee0-bb53-804cca17b9c9', networks=['172.24.5.147/24'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={'hosting-chassis': '3871dd15-d123-44a6-815e-51587164f440'}) old= {{(pid=104232) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Mar 23 22:17:41.970036 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-06cffee1-b8de-433e-a67a-ec0e950c9238 synchronized; highest priority chassis 3871dd15-d123-44a6-815e-51587164f440
Mar 23 22:17:41.970473 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '06cffee1-b8de-433e-a67a-ec0e950c9238', 'neutron:router_id': '30981653-9cf2-456c-b4c8-6836a78525ee'}),), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:41.970824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, chassis=3871dd15-d123-44a6-815e-51587164f440, priority=1, columns={}) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:41.971305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:41.971500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:41.971500 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:41.971500 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:41.971500 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:41.971500 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:41.971500 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:41.971500 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:41.971500 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event Traceback (most recent call last):
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     match.run(event, row, updates)
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 88, in run
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1639, in update_router_ha_chassis_group
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2385, in link_network_ha_chassis_group
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     next(self.gen)
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     next(self.gen)
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:41.972607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with self.create_transaction(
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     self.result = self.commit()
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     raise result.ex
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:41.973162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event 
Mar 23 22:17:42.001618 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-06cffee1-b8de-433e-a67a-ec0e950c9238 synchronized; highest priority chassis 3871dd15-d123-44a6-815e-51587164f440
Mar 23 22:17:42.002424 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '06cffee1-b8de-433e-a67a-ec0e950c9238', 'neutron:router_id': '30981653-9cf2-456c-b4c8-6836a78525ee'}),), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:42.003049 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, chassis=3871dd15-d123-44a6-815e-51587164f440, priority=1, columns={}) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:42.003332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:42.004022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:42.004022 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:42.004022 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:42.004022 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:42.004022 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:42.004022 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:42.004022 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:42.004022 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event Traceback (most recent call last):
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     match.run(event, row, updates)
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 80, in run
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1639, in update_router_ha_chassis_group
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2385, in link_network_ha_chassis_group
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     next(self.gen)
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     next(self.gen)
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:42.005271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with self.create_transaction(
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     self.result = self.commit()
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     raise result.ex
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:42.005826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event 
Mar 23 22:17:44.345314 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:44.347989 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:44.386608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:44.386832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.387435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:17:44.387435 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.387435 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:44.387435 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.387435 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.387435 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:44.387435 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.387435 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.388594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.389166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:44.408948 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:44.410316 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:44.410544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.411156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:44.411156 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.411156 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:44.411156 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.411156 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.411156 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:44.411156 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.411156 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.413055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.414537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:44.567299 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:44.568497 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:17:44.569272 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:44.569517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.570161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:44.570161 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.570161 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:44.570161 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.570161 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.570161 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:44.570161 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.570161 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:17:44.571548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:44.572926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:17:44.572926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.572926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:17:44.572926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.572926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.572926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:17:44.572926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.572926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:17:44.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.574591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.575584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:44.575584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.575584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:44.663576 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:44.664611 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:44.665210 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:44.665612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.666382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:44.666382 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.666382 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:44.666382 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.666382 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.666382 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:44.666382 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.666382 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:17:44.667749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:44.668625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:17:44.668625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.668625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:17:44.668625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.668625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.668625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:17:44.668625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.668625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:17:44.669484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.670278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:44.671152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.671152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:44.712737 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:44.713658 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d41a5c6-8cd0-49da-a3b4-e79b7a5348f2 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:44.714311 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:44.714723 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:44.714993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.715796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d41a5c6-8cd0-49da-a3b4-e79b7a5348f2 None None] Traceback (most recent call last):
Mar 23 22:17:44.715796 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.715796 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:44.715796 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.715796 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.715796 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:44.715796 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.715796 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-9d41a5c6-8cd0-49da-a3b4-e79b7a5348f2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.717185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:44.718052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:49.347033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:49.348850 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:49.384267 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:49.384599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.385388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:17:49.385388 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.385388 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:49.385388 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.385388 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.385388 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:49.385388 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.385388 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.386360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.387181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:49.409901 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:49.411317 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:49.411692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.412609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:49.412609 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.412609 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:49.412609 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.412609 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.412609 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:49.412609 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.412609 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.413633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.414443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:49.570672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:49.571999 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:17:49.572631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:49.572968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:49.575917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.576563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:49.577007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.577618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:49.664601 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:49.665651 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:49.666145 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:49.666389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.666904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:49.666904 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.666904 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:49.666904 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.666904 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.666904 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:49.666904 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.666904 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:17:49.668184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:49.668899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:17:49.668899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.668899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:17:49.668899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.668899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.668899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:17:49.668899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.668899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:17:49.669840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.670562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:49.671310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.671310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:49.714888 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:49.715825 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d41a5c6-8cd0-49da-a3b4-e79b7a5348f2 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:49.716416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:49.716688 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:49.716806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.717436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d41a5c6-8cd0-49da-a3b4-e79b7a5348f2 None None] Traceback (most recent call last):
Mar 23 22:17:49.717436 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.717436 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:49.717436 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.717436 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.717436 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:49.717436 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.717436 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-9d41a5c6-8cd0-49da-a3b4-e79b7a5348f2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.718679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:49.719396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:54.348143 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:54.350238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:54.384010 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:54.384483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.385473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:17:54.385473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.385473 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:54.385473 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.385473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.385473 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:54.385473 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.385473 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.386670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.387692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:54.410749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:54.412397 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:54.412875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.414167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:54.414167 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.414167 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:54.414167 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.414167 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.414167 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:54.414167 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.414167 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.416637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.417443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:54.572250 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:54.573458 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:17:54.574069 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:54.574572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:54.578108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.580434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:54.580757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.581152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:54.665589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:54.667612 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:54.667924 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:54.668130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.668526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:54.668526 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.668526 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:54.668526 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.668526 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.668526 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:54.668526 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.668526 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:17:54.669329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.669791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:54.670325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:54.670892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:54.670892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:54.670892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.670892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:54.670892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.670892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.670892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:54.670892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.670892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:54.715910 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:54.716984 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:54.718139 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:54.719507 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:54.720261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.720929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:54.720929 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.720929 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:54.720929 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.720929 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.720929 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:54.720929 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.720929 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.721729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:54.722095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:59.349777 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:59.352630 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:59.385976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:59.386548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.387565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:17:59.387565 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.387565 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:59.387565 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.387565 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.387565 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:59.387565 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.387565 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.388519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.389731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:59.412195 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:59.413795 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:59.414123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.416663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:59.417661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.418808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.418808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:59.418808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.418808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:59.573438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:59.574369 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:17:59.574745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:59.574949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.575456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:59.575456 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.575456 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:59.575456 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.575456 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.575456 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:59.575456 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.575456 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:17:59.576309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:59.576799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:17:59.576799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.576799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:17:59.576799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.576799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.576799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:17:59.576799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.576799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:17:59.577380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.578393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.578853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:59.578853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.578853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:59.667982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:59.668989 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:59.669557 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:59.669908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.670626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:59.670626 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.670626 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:59.670626 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.670626 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.670626 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:59.670626 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.670626 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:17:59.671935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:59.672627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:17:59.672627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.672627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:17:59.672627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.672627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.672627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:17:59.672627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.672627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:17:59.675484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.676283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:59.676738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.676738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:17:59.717311 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:17:59.718728 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:17:59.719124 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:59.719582 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:17:59.719582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.722560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:17:59.723674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:17:59.724583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:17:59.724583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:17:59.724583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:17:59.724583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:04.351439 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:04.354117 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:04.387594 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:04.387984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.388847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:18:04.388847 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.388847 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:04.388847 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.388847 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.388847 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:04.388847 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.388847 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.389716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.390262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:04.413352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:04.416353 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:04.416353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.417352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:04.417352 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.417352 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:04.417352 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.417352 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.417352 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:04.417352 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.417352 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.418613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.419600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:04.574834 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:04.576613 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:04.577308 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:04.577780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.578709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:04.578709 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.578709 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:04.578709 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.578709 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.578709 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:04.578709 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.578709 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:04.580288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:04.581149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:04.581149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.581149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:04.581149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.581149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.581149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:04.581149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.581149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:04.582111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.583167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.584114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:04.584114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.584114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:04.669319 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:04.670619 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:04.671400 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:04.671842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.672697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:04.672697 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.672697 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:04.672697 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.672697 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.672697 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:04.672697 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.672697 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:04.674117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:04.674898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:04.674898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.674898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:04.674898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.674898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.674898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:04.674898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.674898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:04.675796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.676614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:04.677591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.677591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:04.718360 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:04.720722 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:04.721265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:04.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:04.721863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.722671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:04.722671 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.722671 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:04.722671 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.722671 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.722671 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:04.722671 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.722671 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.723857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:04.724597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:09.353481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:09.356327 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:09.395875 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:09.396165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.396933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:18:09.396933 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.396933 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:09.396933 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.396933 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.396933 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:09.396933 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.396933 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.397856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.398442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:09.414890 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:09.416855 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:09.417130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.418077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:09.418077 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.418077 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:09.418077 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.418077 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.418077 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:09.418077 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.418077 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.419904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.420793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:09.576821 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:09.581131 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:09.581759 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:09.582115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.582913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:09.582913 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.582913 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:09.582913 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.582913 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.582913 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:09.582913 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.582913 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:09.585517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:09.586537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:09.586537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.586537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:09.586537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.586537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.586537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:09.586537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.586537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:09.587762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.588793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.589415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:09.589415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.589415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:09.670549 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:09.673198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:09.673858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:09.674245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.675036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:09.675036 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.675036 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:09.675036 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.675036 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.675036 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:09.675036 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.675036 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:09.676834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:09.677579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:09.677579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.677579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:09.677579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.677579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.677579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:09.677579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.677579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:09.679378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.680523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:09.681726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.681726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:09.721743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:09.722868 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:09.723356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:09.723731 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:09.723927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.724772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:09.724772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.724772 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:09.724772 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.724772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.724772 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:09.724772 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.724772 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.728752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:09.729976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:14.355314 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:14.358517 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:14.389547 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:14.389829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:14.392208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:14.393254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.394298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:14.394298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.394298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.394298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:14.394298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.394298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:14.416320 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:14.418187 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:14.418539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.419455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:14.419455 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.419455 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:14.419455 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.419455 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.419455 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:14.419455 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.419455 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.421427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.422306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:14.580912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:14.582524 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:14.583362 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:14.583603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.584428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:14.584428 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.584428 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:14.584428 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.584428 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.584428 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:14.584428 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.584428 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:14.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:14.588093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:14.588093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.588093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:14.588093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.588093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.588093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:14.588093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.588093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:14.588735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.589196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.589732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:14.589732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.589732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:14.673126 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:14.674373 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:14.674941 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:14.675166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.675782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:14.675782 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.675782 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:14.675782 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.675782 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.675782 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:14.675782 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.675782 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:14.677382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:14.678089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:14.678089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.678089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:14.678089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.678089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.678089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:14.678089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.678089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:14.679235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.680138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:14.681038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.681038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:14.722982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:14.725672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:14.727753 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:14.728018 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:14.728136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.728740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:14.728740 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.728740 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:14.728740 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.728740 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.728740 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:14.728740 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.728740 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.729646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:14.730636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:19.356474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:19.359129 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:19.392463 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:19.392925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.394065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:18:19.394065 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.394065 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:19.394065 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.394065 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.394065 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:19.394065 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.394065 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.395314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.396132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:19.417646 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:19.419464 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:19.419893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.420758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:19.420758 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.420758 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:19.420758 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.420758 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.420758 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:19.420758 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.420758 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.422540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.422867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:19.582242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:19.583113 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:19.583932 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:19.584302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.585044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:19.585044 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.585044 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:19.585044 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.585044 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.585044 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:19.585044 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.585044 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:19.585927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:19.586337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:19.586337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.586337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:19.586337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.586337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.586337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:19.586337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.586337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:19.586980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.587313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.587663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:19.587663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.587663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:19.674338 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:19.675409 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:19.676193 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:19.676694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:19.677705 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.677705 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:19.677705 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.677705 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.677705 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:19.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:19.678954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:19.679815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:19.679815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.679815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:19.679815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.679815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.679815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:19.679815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.679815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:19.680698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.682171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:19.684088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.684088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:19.725658 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:19.726745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:19.727337 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:19.727820 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:19.727986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.730602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:19.731443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:19.732286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:19.732286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:19.732286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:19.732286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:24.358258 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:24.361379 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:24.392402 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:24.392852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:24.394718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:24.395226 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.395605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:24.395605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.395605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.395605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:24.395605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.395605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:24.419210 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:24.420872 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:24.421073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.421865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:24.421865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.421865 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:24.421865 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.421865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.421865 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:24.421865 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.421865 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.423611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.424399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:24.583567 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:24.584790 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:24.585582 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:24.586009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.586985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:24.586985 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.586985 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:24.586985 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.586985 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.586985 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:24.586985 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.586985 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:24.588304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:24.588836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:24.588836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.588836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:24.588836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.588836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.588836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:24.588836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.588836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:24.589870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.590383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.590921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:24.590921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.590921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:24.675501 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:24.676898 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:24.677380 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:24.677748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.678432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:24.678432 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.678432 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:24.678432 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.678432 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.678432 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:24.678432 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.678432 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:24.681936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.682723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:24.683504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:24.684423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:24.684423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:24.684423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.684423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:24.684423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.684423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.684423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:24.684423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.684423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:24.726644 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:24.727794 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:24.728372 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:24.728715 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:24.728878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.729706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:24.729706 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.729706 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:24.729706 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.729706 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.729706 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:24.729706 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.729706 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:24.732004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:29.359639 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:29.362008 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:29.364026 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=577,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:18:29.398347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:29.398764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.399776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:18:29.399776 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.399776 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:29.399776 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.399776 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.399776 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:29.399776 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.399776 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.401752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.402683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:29.420453 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:29.421801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:29.422106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.422873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:29.422873 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.422873 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:29.422873 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.422873 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.422873 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:29.422873 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.422873 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.424549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.425397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:29.585530 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:29.586616 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:29.587523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:29.587948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.589007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:29.589007 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.589007 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:29.589007 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.589007 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.589007 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:29.589007 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.589007 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:29.590406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:29.591161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:29.591161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.591161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:29.591161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.591161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.591161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:29.591161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.591161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:29.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.592698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.593563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:29.593563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.593563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:29.676828 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:29.678195 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:29.678763 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:29.679254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.680088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:29.680088 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.680088 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:29.680088 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.680088 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.680088 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:29.680088 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.680088 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:29.680979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:29.681776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:29.681776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.681776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:29.681776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.681776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.681776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:29.681776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.681776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:29.682193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.683067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:29.683955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.683955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:29.727754 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:29.728770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:29.729347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:29.729704 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:29.729888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.730656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:29.730656 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.730656 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:29.730656 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.730656 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.730656 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:29.730656 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.730656 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.732306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:29.733173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:34.361084 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:34.363274 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:34.395770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:34.396272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.397179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:18:34.397179 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.397179 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:34.397179 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.397179 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.397179 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:34.397179 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.397179 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.398253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.399120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:34.421489 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:34.423252 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:34.423601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.424597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:34.424597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.424597 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:34.424597 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.424597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.424597 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:34.424597 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.424597 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.427509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.428275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:34.586661 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:34.587624 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:34.588099 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:34.588415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.589186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:34.589186 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.589186 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:34.589186 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.589186 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.589186 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:34.589186 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.589186 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:34.590435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:34.591270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:34.591270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.591270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:34.591270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.591270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.591270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:34.591270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.591270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:34.592145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.592964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.593718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:34.593718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.593718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:34.678185 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:34.679388 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:34.680246 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:34.680576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.681426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:34.681426 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.681426 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:34.681426 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.681426 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.681426 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:34.681426 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.681426 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:34.682836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:34.683598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:34.683598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.683598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:34.683598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.683598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.683598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:34.683598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.683598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:34.685170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.686831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:34.688533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.688533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:34.728837 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:34.729794 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:34.730451 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:34.730844 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:34.731006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.732345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:34.732345 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.732345 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:34.732345 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.732345 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.732345 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:34.732345 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.732345 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.733792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:34.734658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:39.362256 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:39.364463 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:39.394730 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:39.394904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.395513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Traceback (most recent call last):
Mar 23 22:18:39.395513 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.395513 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:39.395513 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.395513 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.395513 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:39.395513 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.395513 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.396709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.397686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:39.422715 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:39.423970 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:39.424322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.425094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:39.425094 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.425094 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:39.425094 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.425094 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.425094 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:39.425094 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.425094 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.427252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.428073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:39.588075 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:39.588605 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:39.589165 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:39.589616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.590879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:39.590879 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.590879 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:39.590879 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.590879 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.590879 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:39.590879 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.590879 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:39.592248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:39.592937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:39.592937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.592937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:39.592937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.592937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.592937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:39.592937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.592937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:39.593737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.594251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.594851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:39.594851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.594851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:39.679676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:39.680649 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:39.681431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:39.681831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.682767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:39.682767 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.682767 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:39.682767 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.682767 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.682767 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:39.682767 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.682767 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:39.684052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:39.684812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:39.684812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.684812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:39.684812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.684812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.684812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:39.684812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.684812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:39.685752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.686257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:39.686792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.686792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:39.729863 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:39.730497 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:39.731173 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:39.731585 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:39.731778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.732531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:39.732531 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.732531 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:39.732531 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.732531 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.732531 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:39.732531 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.732531 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.733694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:39.734185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:44.385544 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:44.385983 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:44.414385 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:44.414769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.415767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:18:44.415767 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.415767 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:44.415767 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.415767 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.415767 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:44.415767 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.415767 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.416739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.417644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:44.423048 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:44.424017 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:44.424376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.425335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:44.425335 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.425335 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:44.425335 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.425335 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.425335 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:44.425335 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.425335 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.427250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.427848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:44.588518 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:44.589001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:44.589732 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:44.589971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.590731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:44.590731 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.590731 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:44.590731 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.590731 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.590731 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:44.590731 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.590731 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:44.591828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:44.592771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:44.592771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.592771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:44.592771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.592771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.592771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:44.592771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.592771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:44.593159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.594000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.595009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:44.595009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.595009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:44.680804 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:44.681198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:44.681795 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:44.682151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.683257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:44.683257 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.683257 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:44.683257 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.683257 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.683257 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:44.683257 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.683257 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:44.684883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:44.685722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:44.685722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.685722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:44.685722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.685722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.685722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:44.685722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.685722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:44.686840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.687852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:44.688924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.688924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:44.730688 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:44.731312 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:44.731806 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:44.732325 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:44.732509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.733353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:44.733353 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.733353 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:44.733353 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.733353 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.733353 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:44.733353 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.733353 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.736040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:44.736998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:49.385481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:49.387942 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-67ec6529-fbd8-4078-8beb-ee60c9b18644 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:49.423509 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:49.424714 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:49.424947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.425784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:49.425784 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:49.425784 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:49.425784 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:49.425784 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:49.425784 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:49.425784 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.425784 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.427648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.428732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:49.589078 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:49.589695 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:49.590349 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:49.590578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.591155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:49.591155 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:49.591155 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:49.591155 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:49.591155 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:49.591155 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:49.591155 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.591155 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:49.592095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:49.592939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:49.592939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:49.592939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:49.592939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:49.592939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:49.592939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:49.592939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.592939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:49.593860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:49.594632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:49.595493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:49.595493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.595493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:49.681491 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:49.682036 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:49.682792 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:49.683245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.684133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:49.684133 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:49.684133 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:49.684133 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:49.684133 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:49.684133 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:49.684133 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.684133 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:49.685498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:49.686356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:49.686356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:49.686356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:49.686356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:49.686356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:49.686356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:49.686356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.686356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:49.687174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:49.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:49.689109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.689109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:49.731264 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:49.731918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:49.732942 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:49.732942 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:49.733136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.733889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:49.733889 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:49.733889 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:49.733889 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:49.733889 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:49.733889 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:49.733889 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.733889 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.735631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:49.736541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:50.933182 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:50.934099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:50.934435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-67ec6529-fbd8-4078-8beb-ee60c9b18644 None None] Traceback (most recent call last):
Mar 23 22:18:50.934435 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:50.934435 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:50.934435 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:50.934435 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:50.934435 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:50.934435 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:50.934435 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-67ec6529-fbd8-4078-8beb-ee60c9b18644 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:50.935347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:50.936072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:54.386380 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:54.388171 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:54.424021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:54.441155 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:54.441443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.442978 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:54.442978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.443607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:54.443607 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.443607 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:54.443607 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.443607 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.443607 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:54.443607 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.443607 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:54.444203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:18:54.444203 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.444203 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:54.444203 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.444203 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.444203 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:54.444203 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.444203 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.445632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.446148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.447567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.448051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:54.589579 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:54.590175 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:54.590851 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:54.591103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.591902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:54.591902 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.591902 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:54.591902 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.591902 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.591902 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:54.591902 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.591902 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:54.593555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:54.594014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:54.594014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.594014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:54.594014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.594014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.594014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:54.594014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.594014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:54.596168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.596965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.597821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:54.597821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.597821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:54.681870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:54.682437 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:54.683348 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:54.683776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.684716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:54.684716 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.684716 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:54.684716 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.684716 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.684716 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:54.684716 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.684716 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:54.686161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:54.687088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:54.687088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.687088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:54.687088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.687088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.687088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:54.687088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.687088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:54.688929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.689688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:54.690631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.690631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:54.731685 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:54.732254 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:54.732854 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:54.733303 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:54.733564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.734546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:54.734546 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.734546 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:54.734546 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.734546 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.734546 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:54.734546 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.734546 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.736468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:54.737272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:59.387445 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:59.390013 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:59.424133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:59.424653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.425853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:18:59.425853 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.425853 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:59.425853 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.425853 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.425853 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:59.425853 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.425853 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.427032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.428082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:59.429768 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:59.430512 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:59.430789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.431494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:59.431494 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.431494 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:59.431494 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.431494 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.431494 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:59.431494 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.431494 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.432622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.433082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:59.590644 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:59.591466 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:18:59.592367 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:59.592746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.593700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:59.593700 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.593700 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:59.593700 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.593700 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.593700 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:59.593700 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.593700 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:59.595385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:59.597090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:59.597090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.597090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:59.597090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.597090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.597090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:59.597090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.597090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:18:59.597903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.598554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.599040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:59.599040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.599040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:59.682659 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:59.683116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:59.683745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:59.684352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.685109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:59.685109 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.685109 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:59.685109 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.685109 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.685109 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:59.685109 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.685109 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:18:59.685825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.686987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:59.688296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:59.690097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:59.690097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:59.690097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.690097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:59.690097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.690097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.690097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:59.690097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.690097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:18:59.732238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:18:59.732861 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:18:59.733542 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:59.733974 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:18:59.734260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.735084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:18:59.735084 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.735084 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:18:59.735084 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.735084 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.735084 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:18:59.735084 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.735084 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.736823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:18:59.737773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:04.388195 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:04.389785 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f588b869-684c-4888-b165-2d1e72e8d3a5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:04.430464 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:04.431188 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:04.431483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.432936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f588b869-684c-4888-b165-2d1e72e8d3a5 None None] Traceback (most recent call last):
Mar 23 22:19:04.432936 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.432936 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:04.432936 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.432936 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.432936 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:04.432936 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.432936 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-f588b869-684c-4888-b165-2d1e72e8d3a5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.434675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.435617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:04.436446 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:04.436446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.436446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:04.436446 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.436446 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:04.436446 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.436446 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.436446 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:04.436446 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.436446 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.437274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.438004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:04.591426 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:04.591981 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:04.593136 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:04.593136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.593347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:04.593347 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.593347 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:04.593347 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.593347 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.593347 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:04.593347 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.593347 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:04.594606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:04.595127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:04.595127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.595127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:04.595127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.595127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.595127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:04.595127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.595127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:04.596281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.596761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.597312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:04.597312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.597312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:04.683384 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:04.683384 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:04.684377 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:04.684728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.685415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:04.685415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.685415 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:04.685415 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.685415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.685415 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:04.685415 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.685415 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:04.686417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:04.687297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:04.687297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.687297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:04.687297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.687297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.687297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:04.687297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.687297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:04.687830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.688921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:04.689912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.689912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:04.732768 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:04.733328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:04.733912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:04.734358 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:04.734555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.735360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:04.735360 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.735360 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:04.735360 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.735360 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.735360 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:04.735360 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.735360 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.737026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:04.737888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:09.388863 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:09.389841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:09.422264 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:09.422720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.423601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Traceback (most recent call last):
Mar 23 22:19:09.423601 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.423601 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:09.423601 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.423601 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.423601 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:09.423601 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.423601 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.424527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.425512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:09.432043 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:09.433019 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:09.433392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.434450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:09.434450 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.434450 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:09.434450 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.434450 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.434450 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:09.434450 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.434450 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.436166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.437077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:09.592133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:09.592773 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:09.595623 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:09.595835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.596389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:09.596389 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.596389 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:09.596389 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.596389 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.596389 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:09.596389 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.596389 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:09.597065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:09.597514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:09.597514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.597514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:09.597514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.597514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.597514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:09.597514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.597514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:09.597962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.598351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.598745 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:09.598745 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.598745 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:09.683483 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:09.683789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:09.684163 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:09.684404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.684880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:09.684880 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.684880 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:09.684880 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.684880 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.684880 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:09.684880 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.684880 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:09.685550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.686317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:09.687127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:09.688104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:09.688104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:09.688104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.688104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:09.688104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.688104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.688104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:09.688104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.688104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:09.733240 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:09.733408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:09.733791 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:09.734016 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:09.734130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.734616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:09.734616 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.734616 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:09.734616 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.734616 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.734616 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:09.734616 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.734616 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.735358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:09.735851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:14.389434 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:14.390998 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:14.420763 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:14.421184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.422232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:19:14.422232 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.422232 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:14.422232 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.422232 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.422232 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:14.422232 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.422232 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.423481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.424400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:14.432541 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:14.433521 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:14.433824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.434793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:14.434793 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.434793 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:14.434793 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.434793 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.434793 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:14.434793 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.434793 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.436687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.437509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:14.592756 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:14.593210 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:14.593668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:14.593911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.594681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:14.594681 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.594681 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:14.594681 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.594681 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.594681 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:14.594681 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.594681 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:14.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:14.595908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:14.595908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.595908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:14.595908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.595908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.595908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:14.595908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.595908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:14.596234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.596822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.597376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:14.597376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.597376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:14.684059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:14.684480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:14.684980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:14.685866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.686053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:14.686053 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.686053 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:14.686053 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.686053 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.686053 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:14.686053 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.686053 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:14.687504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:14.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:14.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:14.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:14.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:14.689236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.689738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:14.690265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.690265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:14.733527 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:14.733957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:14.734613 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:14.734613 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:14.734836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.736163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:14.736163 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.736163 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:14.736163 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.736163 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.736163 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:14.736163 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.736163 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.738185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:14.739106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:19.390062 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:19.391885 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:19.428920 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:19.429363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.430493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:19:19.430493 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.430493 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:19.430493 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.430493 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.430493 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:19.430493 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.430493 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.431458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.432484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:19.433496 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:19.434016 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:19.434318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.435004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:19.435004 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.435004 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:19.435004 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.435004 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.435004 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:19.435004 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.435004 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.436632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.437651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:19.593402 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:19.593841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:19.594464 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:19.594729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.595333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:19.595333 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.595333 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:19.595333 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.595333 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.595333 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:19.595333 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.595333 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:19.596151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:19.596644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:19.596644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.596644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:19.596644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.596644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.596644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:19.596644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.596644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:19.597094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.598007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.598812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:19.598812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.598812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:19.684625 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:19.685062 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:19.685832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:19.686223 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.687109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:19.687109 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.687109 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:19.687109 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.687109 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.687109 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:19.687109 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.687109 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:19.688796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:19.689660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:19.689660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.689660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:19.689660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.689660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.689660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:19.689660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.689660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:19.690870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.691825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:19.692810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.692810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:19.733985 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:19.734523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:19.734857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:19.735151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:19.735255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.735775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:19.735775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.735775 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:19.735775 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.735775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.735775 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:19.735775 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.735775 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.736824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:19.737640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:24.390779 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:24.392370 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:24.428525 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:24.429295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.430131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Traceback (most recent call last):
Mar 23 22:19:24.430131 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.430131 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:24.430131 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.430131 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.430131 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:24.430131 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.430131 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.431095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.432007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:24.433639 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:24.434603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:24.434893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.435614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:24.435614 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.435614 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:24.435614 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.435614 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.435614 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:24.435614 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.435614 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.437316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.437811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:24.594073 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:24.594650 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:24.595189 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:24.595535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.596313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:24.596313 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.596313 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:24.596313 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.596313 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.596313 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:24.596313 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.596313 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:24.597603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:24.598584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:24.598584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.598584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:24.598584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.598584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.598584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:24.598584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.598584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:24.599015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.599888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.601020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:24.601020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.601020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:24.685263 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:24.685731 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:24.686431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:24.686786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.687671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:24.687671 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.687671 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:24.687671 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.687671 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.687671 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:24.687671 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.687671 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:24.689036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:24.690102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:24.690102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.690102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:24.690102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.690102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.690102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:24.690102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.690102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:24.690621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.691713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:24.692914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.692914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:24.734660 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:24.735038 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:24.735869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:24.735869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:24.735869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.735966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:24.735966 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.735966 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:24.735966 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.735966 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.735966 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:24.735966 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.735966 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.737072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:24.737547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:29.391651 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:29.394415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:29.396290 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:19:29.419010 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:29.419174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.419662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:19:29.419662 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.419662 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:29.419662 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.419662 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.419662 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:29.419662 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.419662 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.420436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.420916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:29.434157 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:29.435141 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:29.435504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.436293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:29.436293 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.436293 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:29.436293 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.436293 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.436293 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:29.436293 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.436293 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.438035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.438503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:29.594739 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:29.595311 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:29.595918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:29.596193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.596933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:29.596933 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.596933 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:29.596933 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.596933 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.596933 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:29.596933 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.596933 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:29.598321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:29.599441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:29.599441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.599441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:29.599441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.599441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.599441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:29.599441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.599441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:29.600267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.601380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.602725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:29.602725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.602725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:29.686503 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:29.686503 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:29.686703 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:29.687047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.688619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:29.688619 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.688619 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:29.688619 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.688619 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.688619 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:29.688619 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.688619 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:29.689909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:29.690579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:29.690579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.690579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:29.690579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.690579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.690579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:29.690579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.690579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:29.691614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.692307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:29.693057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.693057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:29.735283 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:29.735624 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:29.736404 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:29.739255 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:29.739255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.739255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:29.739255 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.739255 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:29.739255 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.739255 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.739255 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:29.739255 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.739255 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.739603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:29.740036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:34.392306 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:34.394261 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:34.424247 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:34.424723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.425629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:19:34.425629 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.425629 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:34.425629 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.425629 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.425629 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:34.425629 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.425629 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.426421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.427172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:34.434690 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:34.435638 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:34.435931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.436757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:34.436757 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.436757 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:34.436757 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.436757 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.436757 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:34.436757 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.436757 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.437950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.438954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:34.595403 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:34.595755 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:34.596581 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:34.596937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.597900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:34.597900 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.597900 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:34.597900 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.597900 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.597900 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:34.597900 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.597900 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:34.599177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:34.599936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:34.599936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.599936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:34.599936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.599936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.599936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:34.599936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.599936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:34.600742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.602076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.602634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:34.602634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.602634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:34.686365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:34.686817 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:34.687384 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:34.687799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.688575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:34.688575 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.688575 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:34.688575 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.688575 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.688575 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:34.688575 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.688575 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:34.690014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:34.690789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:34.690789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.690789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:34.690789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.690789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.690789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:34.690789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.690789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:34.691725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.692596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:34.693787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.693787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:34.735468 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:34.735919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:34.736478 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:34.736869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:34.737115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.737976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:34.737976 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.737976 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:34.737976 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.737976 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.737976 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:34.737976 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.737976 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.739652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:34.740556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:39.393038 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:39.394235 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:39.427979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:39.428163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.428594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:19:39.428594 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.428594 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:39.428594 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.428594 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.428594 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:39.428594 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.428594 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.429370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.430341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:39.435370 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:39.435782 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:39.435872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.436231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:39.436231 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.436231 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:39.436231 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.436231 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.436231 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:39.436231 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.436231 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.437165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.437748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:39.596295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:39.597076 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:39.597837 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:39.598105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.599007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:39.599007 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.599007 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:39.599007 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.599007 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.599007 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:39.599007 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.599007 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:39.600351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:39.601170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:39.601170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.601170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:39.601170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.601170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.601170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:39.601170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.601170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:39.601828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.602615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.603658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:39.603658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.603658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:39.687049 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:39.687631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:39.688202 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:39.688581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.689266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:39.689266 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.689266 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:39.689266 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.689266 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.689266 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:39.689266 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.689266 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:39.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:39.691512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:39.691512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.691512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:39.691512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.691512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.691512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:39.691512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.691512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:39.692485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.693364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:39.694178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.694178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:39.735910 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:39.736416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:39.736865 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:39.737206 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:39.737389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.738177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:39.738177 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.738177 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:39.738177 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.738177 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.738177 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:39.738177 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.738177 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.739796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:39.740714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:44.393800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:44.394829 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:44.418851 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:44.419307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.420093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:19:44.420093 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.420093 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:44.420093 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.420093 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.420093 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:44.420093 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.420093 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.420979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.421867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:44.435886 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:44.436901 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:44.437182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.437883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:44.437883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.437883 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:44.437883 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.437883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.437883 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:44.437883 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.437883 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.441740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:44.597046 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:44.597649 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:44.598557 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:44.598907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.599822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:44.599822 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.599822 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:44.599822 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.599822 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.599822 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:44.599822 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.599822 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:44.601371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:44.602066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:44.602066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.602066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:44.602066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.602066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.602066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:44.602066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.602066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:44.603048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.603810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.604642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:44.604642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.604642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:44.687645 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:44.687980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:44.688721 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:44.689179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.689958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:44.689958 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.689958 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:44.689958 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.689958 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.689958 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:44.689958 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.689958 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:44.690543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.691291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:44.691988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:44.692932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:44.692932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:44.692932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.692932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:44.692932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.692932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.692932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:44.692932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.692932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:44.736448 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:44.736881 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:44.737456 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:44.737726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:44.737799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.738415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:44.738415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.738415 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:44.738415 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.738415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.738415 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:44.738415 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.738415 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.740095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:44.740891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:49.394738 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:49.396139 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:49.428938 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:49.429415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.430733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:19:49.430733 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.430733 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:49.430733 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.430733 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.430733 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:49.430733 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.430733 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.431750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.432485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:49.436578 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:49.437205 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:49.437398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.438010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:19:49.438010 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.438010 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:49.438010 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.438010 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.438010 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:49.438010 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.438010 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:49.439852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.441120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:49.597765 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:49.598487 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:49.599295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:49.599635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.600522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:49.600522 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.600522 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:49.600522 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.600522 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.600522 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:49.600522 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.600522 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:49.601915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:49.602800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:49.602800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.602800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:49.602800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.602800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.602800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:49.602800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.602800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:49.603485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.604271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.605111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:49.605111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.605111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:49.688345 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:49.688705 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:49.689373 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:49.689798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.690671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:49.690671 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.690671 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:49.690671 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.690671 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.690671 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:49.690671 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.690671 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:49.691836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:49.692881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:49.692881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.692881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:49.692881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.692881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.692881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:49.692881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.692881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:49.693872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:49.696206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.696206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:49.736992 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:49.737568 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:49.738107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:49.738637 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:49.738833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.739666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:49.739666 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.739666 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:49.739666 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.739666 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.739666 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:49.739666 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.739666 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.741460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:49.742202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:54.395430 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:54.397262 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:54.432054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:54.432567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.433828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:19:54.433828 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.433828 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:54.433828 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.433828 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.433828 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:54.433828 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.433828 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.435304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.436156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:54.436895 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:54.437784 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:54.438061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.438944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:19:54.438944 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.438944 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:54.438944 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.438944 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.438944 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:54.438944 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.438944 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:54.440717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.441417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:54.598501 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:54.599441 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:54.600062 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:54.600520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.601273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:54.601273 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.601273 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:54.601273 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.601273 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.601273 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:54.601273 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.601273 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:54.602109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:54.602895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:54.602895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.602895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:54.602895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.602895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.602895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:54.602895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.602895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:54.603301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.604038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.604922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:54.604922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.604922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:54.688922 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:54.689244 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:54.689626 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:54.689996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:54.690690 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.690690 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:54.690690 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.690690 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.690690 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:54.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.690690 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:54.691879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:54.692808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:54.692808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.692808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:54.692808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.692808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.692808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:54.692808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.692808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:54.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.694273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:54.695078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.695078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:54.737438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:54.738345 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:54.738691 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:54.739018 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:54.739203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.740113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:54.740113 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.740113 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:54.740113 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.740113 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.740113 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:54.740113 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.740113 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:54.742645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:59.396178 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:59.397390 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:59.429017 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:59.429459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.430356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:19:59.430356 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.430356 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:59.430356 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.430356 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.430356 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:59.430356 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.430356 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.431471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.432905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:59.437336 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:59.437983 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:59.438422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.438979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:19:59.438979 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.438979 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:59.438979 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.438979 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.438979 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:59.438979 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.438979 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:59.440625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.441092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:59.599066 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:59.599688 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:19:59.600319 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:59.600618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.601332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:59.601332 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.601332 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:59.601332 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.601332 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.601332 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:59.601332 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.601332 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:59.602231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:59.602996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:59.602996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.602996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:59.602996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.602996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.602996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:59.602996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.602996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:19:59.603418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.604175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.605073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:59.605073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.605073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:59.689490 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:59.689777 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:59.690247 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:59.690660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.691421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:59.691421 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.691421 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:59.691421 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.691421 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.691421 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:59.691421 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.691421 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:19:59.692620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:59.693523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:19:59.693523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.693523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:19:59.693523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.693523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.693523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:19:59.693523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.693523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:19:59.694348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.695125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:59.696145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.696145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:19:59.738269 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:19:59.738694 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:19:59.739125 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:59.739545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:19:59.739739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.740479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:19:59.740479 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.740479 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:19:59.740479 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.740479 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.740479 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:19:59.740479 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.740479 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.741640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:19:59.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:04.396886 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:04.398278 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:04.426893 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:04.427024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.427456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:20:04.427456 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.427456 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:04.427456 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.427456 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.427456 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:04.427456 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.427456 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.428240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.429043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:04.437631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:04.438683 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:04.438956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.439901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Traceback (most recent call last):
Mar 23 22:20:04.439901 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.439901 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:04.439901 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.439901 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.439901 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:04.439901 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.439901 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:04.441946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.442734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:04.599787 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:04.600387 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:04.601037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:04.601397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.602194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:04.602194 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.602194 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:04.602194 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.602194 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.602194 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:04.602194 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.602194 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:04.603627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:04.604416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:04.604416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.604416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:04.604416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.604416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.604416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:04.604416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.604416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:04.605251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.606096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.607062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:04.607062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.607062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:04.689976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:04.690356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:04.690990 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:04.691305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.691923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:04.691923 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.691923 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:04.691923 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.691923 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.691923 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:04.691923 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.691923 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:04.693254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:04.694003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:04.694003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.694003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:04.694003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.694003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.694003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:04.694003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.694003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:04.694956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.695878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:04.696639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.696639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:04.738765 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:04.739355 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:04.739905 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:04.740243 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:04.740482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.741240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:04.741240 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.741240 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:04.741240 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.741240 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.741240 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:04.741240 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.741240 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.743004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:04.743905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:09.397584 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:09.398779 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:09.554832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:09.560424 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:09.560726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.562549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:09.562549 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.562549 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:09.562549 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.562549 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.562549 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:09.562549 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.562549 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.565840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.566738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:09.588869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:09.589237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.590020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:20:09.590020 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.590020 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:09.590020 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.590020 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.590020 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:09.590020 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.590020 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.591186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.591791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:09.600239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:09.600591 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:09.600971 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:09.601241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.602027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:09.602027 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.602027 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:09.602027 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.602027 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.602027 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:09.602027 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.602027 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:09.603047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:09.603981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:09.603981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.603981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:09.603981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.603981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.603981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:09.603981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.603981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:09.604561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.605415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.606315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:09.606315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.606315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:09.690740 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:09.691132 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:09.691629 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:09.691964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.692615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:09.692615 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.692615 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:09.692615 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.692615 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.692615 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:09.692615 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.692615 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:09.693747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.694844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:09.695926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:09.697490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:09.697490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:09.697490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.697490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:09.697490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.697490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.697490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:09.697490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.697490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:09.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:09.739777 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:09.740256 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:09.740677 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:09.740903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.741621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:09.741621 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.741621 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:09.741621 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.741621 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.741621 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:09.741621 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.741621 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.742754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:09.743487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:14.398152 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:14.400386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:14.448257 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:14.448694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.449830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:20:14.449830 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.449830 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:14.449830 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.449830 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.449830 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:14.449830 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.449830 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.451324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.452310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:14.559388 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:14.560393 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:14.560626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.561428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:14.561428 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.561428 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:14.561428 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.561428 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.561428 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:14.561428 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.561428 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.563334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.564162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:14.600708 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:14.601103 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:14.601571 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:14.601852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.602744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:14.602744 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.602744 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:14.602744 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.602744 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.602744 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:14.602744 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.602744 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:14.604112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:14.604920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:14.604920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.604920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:14.604920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.604920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.604920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:14.604920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.604920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:14.605696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.606494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.607392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:14.607392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.607392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:14.691245 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:14.691659 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:14.692316 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:14.692629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.693304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:14.693304 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.693304 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:14.693304 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.693304 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.693304 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:14.693304 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.693304 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:14.694194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.695329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:14.696377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:14.697851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:14.697851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:14.697851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.697851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:14.697851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.697851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.697851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:14.697851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.697851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:14.739836 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:14.740558 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:14.741093 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:14.741550 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:14.741740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:14.742556 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.742556 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:14.742556 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.742556 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.742556 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:14.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.744451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:14.745311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:19.398874 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:19.400487 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:19.432965 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:19.433412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:19.434355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:20:19.434355 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:19.434355 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:19.434355 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:19.434355 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:19.434355 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:19.434355 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:19.434355 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:19.435532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:19.436604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.033648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.035103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:20.036332 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:20.037192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:20.037983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:20.038931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:20.039823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:20.040880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:20.040880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:20.040880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:20.040880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:20.040880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:20.040880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.040880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:20.040880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:20.040880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:20.040880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:20.040880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:20.041580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.042663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:20.043554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:20.044646 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.045717 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.046187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:20.046579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:24.399516 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:24.401009 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:24.432571 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:24.432811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.433332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:20:24.433332 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.433332 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:24.433332 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.433332 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.433332 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:24.433332 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.433332 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.434279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.435271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:24.560974 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:24.561899 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:24.562192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.563068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:24.563068 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.563068 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:24.563068 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.563068 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.563068 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:24.563068 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.563068 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.564782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.565564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:24.601827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:24.602431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:24.603063 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:24.603367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.604091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:24.604091 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.604091 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:24.604091 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.604091 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.604091 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:24.604091 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.604091 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:24.605978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:24.606719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:24.606719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.606719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:24.606719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.606719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.606719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:24.606719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.606719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:24.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.609103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.609961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:24.609961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.609961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:24.693002 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:24.693661 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:24.694244 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:24.694689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.695462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:24.695462 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.695462 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:24.695462 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.695462 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.695462 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:24.695462 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.695462 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:24.696787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:24.697525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:24.697525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.697525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:24.697525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.697525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.697525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:24.697525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.697525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:24.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.699461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:24.700281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.700281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:24.741054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:24.741701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:24.742205 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:24.742581 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:24.742805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.743556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:24.743556 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.743556 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:24.743556 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.743556 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.743556 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:24.743556 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.743556 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.745181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:24.745966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:29.400179 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:29.401885 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:29.403775 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:20:29.434121 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:29.434581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.435481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:20:29.435481 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.435481 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:29.435481 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.435481 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.435481 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:29.435481 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.435481 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.436419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.437172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:29.561488 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:29.562700 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:29.562971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.564041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:29.564041 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.564041 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:29.564041 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.564041 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.564041 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:29.564041 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.564041 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.565915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.566363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:29.602388 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:29.602915 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:29.603495 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:29.603791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.604623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:29.604623 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.604623 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:29.604623 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.604623 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.604623 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:29.604623 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.604623 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:29.605888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:29.606640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:29.606640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.606640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:29.606640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.606640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.606640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:29.606640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.606640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:29.607435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.608127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.609028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:29.609028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.609028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:29.693538 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:29.693827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:29.694229 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:29.694630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.695251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:29.695251 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.695251 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:29.695251 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.695251 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.695251 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:29.695251 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.695251 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:29.696491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.701120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:29.701873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:29.702791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:29.702791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:29.702791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.702791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:29.702791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.702791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.702791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:29.702791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.702791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:29.741738 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:29.742196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:29.742730 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:29.743112 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:29.743347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.744134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:29.744134 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.744134 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:29.744134 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.744134 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.744134 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:29.744134 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.744134 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.745720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:29.746537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:34.400889 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:34.402702 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:34.436868 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:34.437303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.438343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:20:34.438343 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.438343 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:34.438343 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.438343 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.438343 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:34.438343 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.438343 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.439525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.440335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:34.562161 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:34.563241 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:34.563609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.564569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f588b869-684c-4888-b165-2d1e72e8d3a5 None None] Traceback (most recent call last):
Mar 23 22:20:34.564569 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.564569 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:34.564569 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.564569 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.564569 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:34.564569 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.564569 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-f588b869-684c-4888-b165-2d1e72e8d3a5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:34.566471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.567256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:34.602913 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:34.603489 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:34.604014 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:34.604410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.605244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:34.605244 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.605244 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:34.605244 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.605244 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.605244 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:34.605244 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.605244 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:34.606572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:34.607292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:34.607292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.607292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:34.607292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.607292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.607292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:34.607292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.607292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:34.608069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.608792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.609666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:34.609666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.609666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:34.694132 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:34.694568 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:34.695059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:34.695578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.696511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:34.696511 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.696511 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:34.696511 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.696511 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.696511 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:34.696511 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.696511 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:34.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:34.698477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:34.698477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.698477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:34.698477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.698477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.698477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:34.698477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.698477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:34.699744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.700316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:34.700968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.700968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:34.742196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:34.742714 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:34.743356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:34.743773 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:34.743966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.744950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:34.744950 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.744950 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:34.744950 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.744950 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.744950 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:34.744950 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.744950 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.746690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:34.747525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:39.401785 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:39.403652 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:39.434265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:39.434587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.435241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:20:39.435241 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.435241 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:39.435241 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.435241 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.435241 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:39.435241 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.435241 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.436235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.436763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:39.562938 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:39.564031 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:39.564342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.565254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f588b869-684c-4888-b165-2d1e72e8d3a5 None None] Traceback (most recent call last):
Mar 23 22:20:39.565254 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.565254 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:39.565254 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.565254 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.565254 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:39.565254 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.565254 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-f588b869-684c-4888-b165-2d1e72e8d3a5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:39.567116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.567939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:39.603496 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:39.603819 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:39.604422 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:39.604985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.605877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:39.605877 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.605877 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:39.605877 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.605877 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.605877 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:39.605877 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.605877 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:39.607156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:39.608157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:39.608157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.608157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:39.608157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.608157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.608157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:39.608157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.608157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:39.608810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.609420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.610162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:39.610162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.610162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:39.695067 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:39.695636 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:39.696601 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:39.697429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.698476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:39.698476 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.698476 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:39.698476 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.698476 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.698476 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:39.698476 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.698476 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:39.699707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:39.700487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:39.700487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.700487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:39.700487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.700487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.700487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:39.700487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.700487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:39.701526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.702363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:39.703220 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.703220 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:39.742950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:39.743536 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:39.744167 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:39.744773 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:39.745067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.746440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:39.746440 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.746440 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:39.746440 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.746440 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.746440 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:39.746440 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.746440 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.748530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:39.749339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:44.402840 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:44.404463 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:44.443758 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:44.444098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.444832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:20:44.444832 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.444832 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:44.444832 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.444832 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.444832 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:44.444832 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.444832 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.447683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.448578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:44.563666 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:44.564914 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:44.565346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.566442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Traceback (most recent call last):
Mar 23 22:20:44.566442 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.566442 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:44.566442 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.566442 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.566442 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:44.566442 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.566442 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:44.567725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.568491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:44.604139 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:44.604616 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:44.605557 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:44.605916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:44.610018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.611098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:44.611589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.612107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:44.695420 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:44.696030 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:44.696796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:44.697246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.698156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:44.698156 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.698156 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:44.698156 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.698156 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.698156 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:44.698156 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.698156 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:44.699105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:44.699893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:44.699893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.699893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:44.699893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.699893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.699893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:44.699893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.699893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:44.700440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.701215 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:44.702132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.702132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:44.743568 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:44.743870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:44.744441 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:44.744731 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:44.744848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.747260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:44.748248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:44.749111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:44.749111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:44.749111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:44.749111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:49.403447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:49.405287 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:49.438511 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:49.438949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.439879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:20:49.439879 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.439879 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:49.439879 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.439879 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.439879 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:49.439879 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.439879 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.441249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.442250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:49.564313 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:49.566209 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:49.566558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.567460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:20:49.567460 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.567460 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:49.567460 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.567460 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.567460 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:49.567460 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.567460 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:49.569058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.569544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:49.604754 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:49.605153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:49.605730 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:49.605998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.606741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:49.606741 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.606741 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:49.606741 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.606741 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.606741 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:49.606741 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.606741 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:49.608061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:49.608774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:49.608774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.608774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:49.608774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.608774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.608774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:49.608774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.608774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:49.609694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.610437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.611265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:49.611265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.611265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:49.695946 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:49.696437 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:49.697079 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:49.697554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.698328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:49.698328 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.698328 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:49.698328 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.698328 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.698328 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:49.698328 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.698328 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:49.699552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:49.700238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:49.700238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.700238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:49.700238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.700238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.700238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:49.700238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.700238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:49.701170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.701885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:49.702728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.702728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:49.744013 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:49.744349 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:49.744803 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:49.745183 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:49.745363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.746123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:49.746123 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.746123 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:49.746123 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.746123 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.746123 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:49.746123 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.746123 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.747688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:49.748420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:54.404263 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:54.405286 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:54.429766 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:54.430098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.430809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:20:54.430809 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.430809 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:54.430809 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.430809 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.430809 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:54.430809 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.430809 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.431631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.432744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:54.565776 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:54.566864 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:54.567163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.568723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:54.568723 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.568723 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:54.568723 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.568723 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.568723 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:54.568723 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.568723 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.570240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.570695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:54.605323 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:54.605704 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:54.606238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:54.606567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.607305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:54.607305 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.607305 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:54.607305 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.607305 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.607305 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:54.607305 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.607305 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:54.608579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:54.609859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:54.609859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.609859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:54.609859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.609859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.609859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:54.609859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.609859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:54.610414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.611448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.612550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:54.612550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.612550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:54.696526 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:54.696961 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:54.697733 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:54.697967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.698579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:54.698579 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.698579 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:54.698579 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.698579 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.698579 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:54.698579 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.698579 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:54.699880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:54.700576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:54.700576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.700576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:54.700576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.700576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.700576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:54.700576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.700576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:54.702007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.702715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:54.703532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.703532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:54.744489 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:54.744856 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:54.745443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:54.745873 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:54.746059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.746789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:54.746789 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.746789 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:54.746789 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.746789 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.746789 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:54.746789 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.746789 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.747886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:54.748621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:59.404872 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:59.406617 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:59.439771 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:59.440249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.441229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:20:59.441229 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.441229 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:59.441229 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.441229 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.441229 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:59.441229 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.441229 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.442458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.443388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:59.566606 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:59.567717 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:59.567992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.569028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:59.569028 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.569028 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:59.569028 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.569028 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.569028 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:59.569028 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.569028 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.570821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.571628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:59.605883 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:59.606428 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:20:59.607055 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:59.607476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.608352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:59.608352 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.608352 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:59.608352 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.608352 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.608352 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:59.608352 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.608352 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:59.609846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:59.610670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:59.610670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.610670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:59.610670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.610670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.610670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:59.610670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.610670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:20:59.611960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.612700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.613559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:59.613559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.613559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:59.696936 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:59.697336 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:59.697765 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:59.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.698914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:59.698914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.698914 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:59.698914 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.698914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.698914 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:59.698914 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.698914 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:20:59.699981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:59.700688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:20:59.700688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.700688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:20:59.700688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.700688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.700688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:20:59.700688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.700688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:20:59.701597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.702419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:59.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:20:59.745015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:20:59.745475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:20:59.746122 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:59.746351 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:20:59.746581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.747356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:20:59.747356 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.747356 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:20:59.747356 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.747356 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.747356 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:20:59.747356 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.747356 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.749132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:20:59.749966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:04.406441 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:04.409534 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:04.448910 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:04.449670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.450955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:04.450955 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.450955 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:04.450955 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.450955 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.450955 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:04.450955 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.450955 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.451260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.451585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:04.567275 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:04.568570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:04.568892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.570266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:04.570266 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.570266 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:04.570266 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.570266 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.570266 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:04.570266 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.570266 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.572055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.572804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:04.606414 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:04.606810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:04.607593 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:04.607934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.608729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:04.608729 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.608729 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:04.608729 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.608729 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.608729 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:04.608729 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.608729 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:04.609652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:04.610450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:04.610450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.610450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:04.610450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.610450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.610450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:04.610450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.610450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:04.610802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.612333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.614386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:04.614386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.614386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:04.697665 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:04.698025 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:04.698629 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:04.699014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.699737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:04.699737 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.699737 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:04.699737 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.699737 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.699737 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:04.699737 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.699737 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:04.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:04.701774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:04.701774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.701774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:04.701774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.701774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.701774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:04.701774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.701774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:04.702361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.703300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:04.704336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.704336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:04.745670 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:04.746113 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:04.746882 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:04.747116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:04.747401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.748270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:04.748270 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.748270 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:04.748270 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.748270 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.748270 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:04.748270 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.748270 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.749631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:04.750078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:09.408257 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:09.409843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:09.441566 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:09.441795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.442389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:09.442389 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.442389 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:09.442389 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.442389 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.442389 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:09.442389 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.442389 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.443425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.444334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:09.568024 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:09.568944 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:09.569333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.570148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:21:09.570148 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.570148 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:09.570148 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.570148 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.570148 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:09.570148 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.570148 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:09.571256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.572041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:09.606988 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:09.607346 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:09.607845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:09.608143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.608986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:09.608986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.608986 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:09.608986 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.608986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.608986 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:09.608986 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.608986 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:09.610180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:09.611171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:09.611171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.611171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:09.611171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.611171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.611171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:09.611171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.611171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:09.612069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.612979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.614269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:09.614269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.614269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:09.698375 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:09.698701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:09.699048 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:09.699538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.700475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:09.700475 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.700475 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:09.700475 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.700475 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.700475 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:09.700475 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.700475 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:09.702739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:09.702739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.702739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:09.702739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.702739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.702739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:09.702739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.702739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:09.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.704949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:09.706136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.706136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:09.746089 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:09.746296 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:09.746588 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:09.746827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:09.746935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.747452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:09.747452 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.747452 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:09.747452 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.747452 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.747452 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:09.747452 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.747452 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.748479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:09.749602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:14.409000 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:14.410762 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:14.441369 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:14.441762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.442578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:14.442578 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.442578 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:14.442578 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.442578 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.442578 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:14.442578 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.442578 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.443455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.444171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:14.568638 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:14.570036 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:14.570338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.570908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:21:14.570908 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.570908 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:14.570908 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.570908 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.570908 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:14.570908 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.570908 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:14.571982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.572688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:14.607503 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:14.607807 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:14.608331 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:14.608631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.609299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:14.609299 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.609299 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:14.609299 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.609299 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.609299 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:14.609299 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.609299 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:14.610420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:14.611070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:14.611070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.611070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:14.611070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.611070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.611070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:14.611070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.611070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:14.611917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.613305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:14.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:14.698729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:14.699019 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:14.699509 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:14.699879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.700604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:14.700604 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.700604 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:14.700604 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.700604 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.700604 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:14.700604 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.700604 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:14.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:14.702382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:14.702382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.702382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:14.702382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.702382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.702382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:14.702382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.702382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:14.703050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.703729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:14.704521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.704521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:14.746494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:14.747079 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:14.747526 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:14.747882 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:14.748072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.748884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:14.748884 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.748884 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:14.748884 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.748884 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.748884 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:14.748884 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.748884 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.750838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:14.751591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:19.409821 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:19.411097 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:19.448873 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:19.449019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.449547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:19.449547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.449547 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:19.449547 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.449547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.449547 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:19.449547 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.449547 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.450389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.451181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:19.569347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:19.570585 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:19.571128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.572312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:21:19.572312 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.572312 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:19.572312 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.572312 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.572312 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:19.572312 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.572312 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:19.574497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.575255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:19.608208 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:19.608799 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:19.609559 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:19.609913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.610710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:19.610710 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.610710 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:19.610710 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.610710 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.610710 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:19.610710 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.610710 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:19.611902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:19.612816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:19.612816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.612816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:19.612816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.612816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.612816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:19.612816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.612816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:19.613600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.614687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.615379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:19.615379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.615379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:19.699383 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:19.699558 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:19.700040 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:19.700469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.701272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:19.701272 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.701272 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:19.701272 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.701272 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.701272 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:19.701272 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.701272 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:19.702469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:19.703193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:19.703193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.703193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:19.703193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.703193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.703193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:19.703193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.703193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:19.704337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.705021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:19.705772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.705772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:19.746870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:19.746985 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:19.748244 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:19.748719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:19.748925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.749834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:19.749834 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.749834 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:19.749834 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.749834 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.749834 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:19.749834 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.749834 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.751406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:19.752131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:24.410335 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:24.411563 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:24.443026 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:24.443710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:24.446013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:24.446995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.447950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:24.447950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.447950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.447950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:24.447950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.447950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:24.569992 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:24.571435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:24.571786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.573342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Traceback (most recent call last):
Mar 23 22:21:24.573342 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.573342 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:24.573342 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.573342 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.573342 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:24.573342 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.573342 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:24.575649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.576414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:24.608830 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:24.609152 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:24.609726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:24.609962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.610588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:24.610588 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.610588 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:24.610588 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.610588 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.610588 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:24.610588 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.610588 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:24.611252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.612126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:24.612920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:24.613854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:24.613854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:24.613854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:24.613854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:24.613854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.613854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:24.613854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.613854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.613854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:24.613854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.613854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:24.699755 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:24.700056 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:24.700884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:24.701350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.702254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:24.702254 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.702254 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:24.702254 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.702254 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.702254 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:24.702254 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.702254 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:24.703556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:24.703896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:24.703896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.703896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:24.703896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.703896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.703896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:24.703896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.703896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:24.705131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.705508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:24.705882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.705882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:24.747285 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:24.747608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:24.748158 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:24.748608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:24.748869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.749810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:24.749810 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.749810 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:24.749810 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.749810 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.749810 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:24.749810 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.749810 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.751385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:24.751736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:29.410823 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:29.412317 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:29.433367 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=249 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:21:29.453607 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:29.453772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.454261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:29.454261 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.454261 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:29.454261 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.454261 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.454261 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:29.454261 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.454261 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.455070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.455915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:29.571144 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:29.571763 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:29.571994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.572724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:29.572724 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.572724 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:29.572724 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.572724 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.572724 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:29.572724 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.572724 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.574600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.575631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:29.609456 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:29.609780 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:29.611726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:29.612016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:29.614797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.615507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:29.615959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.616735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:29.700348 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:29.700600 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:29.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:29.701695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.702166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:29.702166 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.702166 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:29.702166 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.702166 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.702166 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:29.702166 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.702166 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:29.702923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:29.705854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:29.706958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:29.706958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:29.706958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.706958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:29.706958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.706958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.706958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:29.706958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.706958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:29.748058 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:29.748659 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:29.749181 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:29.749624 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:29.749776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.750627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:29.750627 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.750627 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:29.750627 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.750627 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.750627 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:29.750627 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.750627 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.752516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:29.753706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:34.412187 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:34.413530 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:34.444555 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:34.444930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.445617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:34.445617 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.445617 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:34.445617 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.445617 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.445617 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:34.445617 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.445617 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.447445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.448071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:34.571359 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:34.572197 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:34.572413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.573011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:21:34.573011 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.573011 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:34.573011 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.573011 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.573011 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:34.573011 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.573011 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:34.574555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.574926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:34.609972 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:34.610366 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:34.611008 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:34.611324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.611943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:34.611943 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.611943 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:34.611943 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.611943 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.611943 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:34.611943 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.611943 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:34.612799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:34.613584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:34.613584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.613584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:34.613584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.613584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.613584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:34.613584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.613584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:34.613946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.614861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.615743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:34.615743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.615743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:34.701440 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:34.702046 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:34.702829 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:34.703476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.706184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:34.706184 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.706184 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:34.706184 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.706184 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.706184 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:34.706184 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.706184 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:34.707741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:34.708546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:34.708546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.708546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:34.708546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.708546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.708546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:34.708546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.708546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.709702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:34.710335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.710335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:34.748618 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:34.749017 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:34.749484 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:34.749851 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:34.750065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.750810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:34.750810 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.750810 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:34.750810 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.750810 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.750810 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:34.750810 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.750810 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.752459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:34.752977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:37.497024 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=4f3cc659-b92a-4495-acad-30eee1c1994c, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-bb9d5c47-4694-42c2-a018-18e3c9058915', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-154c08d6-78c5-49fd-a785-0df3f4277a23', 'neutron:subnet_ids': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a a53179c9-417a-4560-9bc7-e3f64551b15b'}, gateway_chassis=[9a8629a4-bda0-4f8c-af6f-28a6834639f4], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:aa:0e:57', name='lrp-348aba7c-88a8-4a19-9486-df3f7edb4007', networks=['172.24.5.146/24', '2001:db8::396/64'], options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=104232) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Mar 23 22:21:39.412985 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:39.414592 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:39.445866 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:39.446304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.447139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:39.447139 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.447139 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:39.447139 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.447139 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.447139 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:39.447139 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.447139 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.447999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.448955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:39.571850 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:39.573338 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:39.573681 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:39.573874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.574500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:21:39.574500 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.574500 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:39.574500 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.574500 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.574500 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:39.574500 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.574500 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:39.575648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.576504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:39.610473 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:39.611041 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:39.612394 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:39.612586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.613516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:39.613516 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.613516 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:39.613516 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.613516 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.613516 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:39.613516 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.613516 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:39.614845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:39.615708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:39.615708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.615708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:39.615708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.615708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.615708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:39.615708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.615708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:39.616385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.617413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.618364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:39.618364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.618364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:39.702137 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:39.702735 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:39.703349 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:39.703713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.704485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:39.704485 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.704485 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:39.704485 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.704485 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.704485 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:39.704485 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.704485 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:39.705978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:39.706845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:39.706845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.706845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:39.706845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.706845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.706845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:39.706845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.706845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:39.707697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.708500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:39.709622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.709622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:39.749167 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:39.749746 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:39.750384 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:39.750733 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:39.750963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.751725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:39.751725 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.751725 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:39.751725 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.751725 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.751725 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:39.751725 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.751725 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.753518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:39.754430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:40.497827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=de29546f-32d9-42a3-abec-df85b86ef5dc, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-154c08d6-78c5-49fd-a785-0df3f4277a23', 'neutron:subnet_ids': '421749b7-bb0d-4854-b3f4-f0414f4c37fb'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:d3:b0:08', name='lrp-d50e2a80-1d68-4882-83e9-3744ac2cf888', networks=['192.168.24.1/24'], options={}, peer=[], status={}) old= {{(pid=104232) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Mar 23 22:21:40.537257 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36 synchronized; highest priority chassis 3871dd15-d123-44a6-815e-51587164f440
Mar 23 22:21:40.538058 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': 'c9e6c30a-b30e-473b-b06a-50be75c90b36', 'neutron:router_id': '154c08d6-78c5-49fd-a785-0df3f4277a23'}}) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:40.538845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, chassis=3871dd15-d123-44a6-815e-51587164f440, priority=1, columns={}) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:40.539190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:40.539995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:40.539995 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:40.539995 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:40.539995 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:40.539995 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:40.539995 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:40.539995 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:40.539995 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event Traceback (most recent call last):
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     match.run(event, row, updates)
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 85, in run
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2385, in link_network_ha_chassis_group
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     next(self.gen)
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     next(self.gen)
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     with self.create_transaction(
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     self.result = self.commit()
Mar 23 22:21:40.541136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Mar 23 22:21:40.541813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:40.541813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     raise result.ex
Mar 23 22:21:40.541813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:40.541813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Mar 23 22:21:40.541813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Mar 23 22:21:40.541813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:40.541813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Mar 23 22:21:40.541813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:40.541813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.event 
Mar 23 22:21:44.413744 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:44.415595 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:44.447150 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:44.447582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.448449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:44.448449 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.448449 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:44.448449 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.448449 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.448449 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:44.448449 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.448449 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.449339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.450111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:44.572628 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:44.573700 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:44.574024 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:44.574348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.575051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:21:44.575051 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.575051 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:44.575051 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.575051 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.575051 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:44.575051 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.575051 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:44.576765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.577660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:44.611012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:44.611411 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:44.611976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:44.612332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.613075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:44.613075 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.613075 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:44.613075 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.613075 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.613075 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:44.613075 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.613075 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:44.614561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:44.615362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:44.615362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.615362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:44.615362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.615362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.615362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:44.615362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.615362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:44.616096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.616887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.617726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:44.617726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.617726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:44.702578 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:44.702974 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:44.703396 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:44.703656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.704146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:44.704146 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.704146 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:44.704146 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.704146 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.704146 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:44.704146 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.704146 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:44.704844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:44.705323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:44.705323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.705323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:44.705323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.705323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.705323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:44.705323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.705323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:44.705591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.706035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:44.706535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.706535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:44.749653 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:44.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:44.750795 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:44.751182 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:44.751465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.752172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:44.752172 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.752172 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:44.752172 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.752172 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.752172 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:44.752172 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.752172 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.753871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:44.754204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:49.414448 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:49.416074 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:49.454334 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:49.454729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.455628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:49.455628 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.455628 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:49.455628 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.455628 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.455628 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:49.455628 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.455628 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.456530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.457264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:49.573272 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:49.574442 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:49.574843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:49.575097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.576006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:21:49.576006 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.576006 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:49.576006 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.576006 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.576006 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:49.576006 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.576006 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:49.577582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.578306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:49.611495 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:49.612021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:49.612617 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:49.612930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.613710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:49.613710 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.613710 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:49.613710 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.613710 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.613710 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:49.613710 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.613710 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:49.614998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:49.615791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:49.615791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.615791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:49.615791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.615791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.615791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:49.615791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.615791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:49.616619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.617329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.618291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:49.618291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.618291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:49.703142 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:49.703726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:49.704394 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:49.704874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.705705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:49.705705 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.705705 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:49.705705 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.705705 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.705705 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:49.705705 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.705705 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:49.707101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:49.708077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:49.708077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.708077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:49.708077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.708077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.708077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:49.708077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.708077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:49.709122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.710142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:49.711035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.711035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:49.750270 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:49.750833 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:49.751452 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:49.751815 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:49.752092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.752918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:49.752918 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.752918 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:49.752918 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.752918 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.752918 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:49.752918 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.752918 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.754645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:49.755628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:54.415271 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:54.416964 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:54.449668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:54.450109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.451039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:54.451039 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.451039 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:54.451039 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.451039 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.451039 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:54.451039 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.451039 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.452041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.452833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:54.573993 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:54.575112 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:54.575487 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:54.575731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.576788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:21:54.576788 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.576788 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:54.576788 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.576788 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.576788 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:54.576788 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.576788 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:54.578695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.579666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:54.612037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:54.612662 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:54.613157 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:54.613481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.614338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:21:54.614338 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.614338 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:54.614338 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.614338 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.614338 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:54.614338 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.614338 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:54.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:54.616408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:54.616408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:54.616408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.616408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:54.616408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.616408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.616408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:54.616408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.616408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:54.617255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.618015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.618944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:54.618944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.618944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:54.703868 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:54.704249 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:54.704853 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:54.705398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.706359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:54.706359 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.706359 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:54.706359 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.706359 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.706359 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:54.706359 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.706359 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:54.707465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:54.708273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:54.708273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.708273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:54.708273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.708273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.708273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:54.708273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.708273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:54.708992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.709756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:54.710798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.710798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:54.753617 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:54.754095 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:54.754636 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:54.755205 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:54.755483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.756265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:54.756265 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.756265 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:54.756265 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.756265 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.756265 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:54.756265 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.756265 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.757571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:54.758094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:59.415987 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:59.417519 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:59.444432 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:59.444859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.445676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:21:59.445676 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.445676 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:59.445676 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.445676 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.445676 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:59.445676 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.445676 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.446589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.447435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:59.574546 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:59.575857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:59.576201 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:59.576509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.577491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:21:59.577491 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.577491 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:59.577491 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.577491 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.577491 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:59.577491 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.577491 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:59.579234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.579934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:59.612564 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:59.613140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:21:59.613739 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:59.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.614959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:21:59.614959 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.614959 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:59.614959 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.614959 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.614959 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:59.614959 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.614959 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:59.616635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:59.617332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:59.617332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:59.617332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.617332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:59.617332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.617332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.617332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:59.617332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.617332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:21:59.618720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.619437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.620274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:59.620274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.620274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:59.704430 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:59.704678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:59.705311 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:59.705666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.706469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:59.706469 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.706469 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:59.706469 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.706469 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.706469 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:59.706469 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.706469 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:21:59.707683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:59.708458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:21:59.708458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.708458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:21:59.708458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.708458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.708458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:21:59.708458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.708458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:21:59.709390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.710118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:59.710916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.710916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:21:59.754117 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:21:59.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:21:59.754972 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:59.755352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:21:59.755576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.756316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:21:59.756316 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.756316 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:21:59.756316 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.756316 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.756316 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:21:59.756316 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.756316 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.757812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:21:59.758595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:04.416783 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:04.418928 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:04.452393 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:04.452590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.453718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:04.453718 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.453718 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:04.453718 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.453718 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.453718 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:04.453718 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.453718 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.454565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.455363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:04.575061 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:04.576779 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:04.577156 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:04.577430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.579476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:22:04.579476 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.579476 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:04.579476 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.579476 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.579476 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:04.579476 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.579476 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:04.581186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.582601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:04.613151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:04.613908 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:04.615386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:04.615610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.619245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.619790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:04.620147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.620586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.621063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:04.621063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.621063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:04.704869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:04.706091 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:04.706690 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:04.706962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.707504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:04.707504 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.707504 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:04.707504 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.707504 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.707504 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:04.707504 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.707504 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:04.708647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.709912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:04.711106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:04.713362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:04.713362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:04.713362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.713362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:04.713362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.713362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.713362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:04.713362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.713362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:04.754864 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:04.755635 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:04.755635 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:04.755923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:04.756072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.757486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:04.757486 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.757486 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:04.757486 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.757486 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.757486 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:04.757486 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.757486 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.758372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:04.758867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:09.418326 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:09.419057 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:09.471397 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.474244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:09.475192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:09.476099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.476099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:09.476099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.476099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.476099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:09.476099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.476099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:09.575734 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:09.578745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:09.578745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:09.578745 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.579530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:22:09.579530 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.579530 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:09.579530 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.579530 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.579530 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:09.579530 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.579530 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:09.580910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.585468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:09.613731 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:09.615197 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:09.617196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:09.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.621653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:22:09.621653 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.621653 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:09.621653 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.621653 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.621653 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:09.621653 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.621653 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:09.622464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:09.623252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:09.623252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:09.623252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.623252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:09.623252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.623252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.623252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:09.623252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.623252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:09.623674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.624519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.625613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:09.625613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.625613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:09.706205 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:09.706755 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:09.707183 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:09.707496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.707998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:09.707998 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.707998 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:09.707998 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.707998 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.707998 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:09.707998 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.707998 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:09.708659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.709594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:09.710355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:09.711284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:09.711284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:09.711284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.711284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:09.711284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.711284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.711284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:09.711284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.711284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:09.755376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:09.756011 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:09.756836 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:09.757243 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:09.757501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.758297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:09.758297 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.758297 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:09.758297 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.758297 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.758297 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:09.758297 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.758297 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.760096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:09.760860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.959404 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.961951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.962959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.963889 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:14.964621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.965462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.966532 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:14.967160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.968062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:14.968989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:14.970116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.970707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.971110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.971478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:14.971811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.972224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.972224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:14.972224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.972224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:14.972224 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:14.972224 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:14.972224 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:14.972224 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:14.972224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.972507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:14.972899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:14.973307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:14.973307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:14.973307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:14.973307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:19.418786 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:19.420129 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:19.462366 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:19.462566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.463206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:19.463206 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.463206 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:19.463206 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.463206 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.463206 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:19.463206 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.463206 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.464182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.465104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:19.577181 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:19.579722 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:19.580184 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:19.580557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.581901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:22:19.581901 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.581901 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:19.581901 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.581901 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.581901 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:19.581901 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.581901 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:19.584498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:19.615747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:19.616135 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:19.616991 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:19.617515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.618790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:19.618790 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.618790 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:19.618790 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.618790 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.618790 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:19.618790 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.618790 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:19.619959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:19.620481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:19.620481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.620481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:19.620481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.620481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.620481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:19.620481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.620481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:19.621451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.621895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.622447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:19.622447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.622447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:19.707435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:19.708143 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:19.709252 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:19.709871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.710862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:19.710862 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.710862 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:19.710862 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.710862 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.710862 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:19.710862 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.710862 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:19.712310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:19.713218 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:19.713218 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.713218 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:19.713218 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.713218 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.713218 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:19.713218 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.713218 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:19.714442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.715324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:19.716365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.716365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:19.756511 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:19.756511 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:19.756913 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:19.757200 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:19.757397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.757936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:19.757936 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.757936 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:19.757936 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.757936 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.757936 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:19.757936 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.757936 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.758980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:19.759780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:24.419598 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:24.421052 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:24.463603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:24.463807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.464269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:24.464269 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.464269 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:24.464269 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.464269 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.464269 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:24.464269 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.464269 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.465039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.466357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:24.578054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:24.584747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:24.585050 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:24.585333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.586129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:22:24.586129 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.586129 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:24.586129 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.586129 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.586129 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:24.586129 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.586129 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:24.587243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.588281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:24.616560 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:24.617282 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:24.617965 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:24.618322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.619136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:24.619136 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.619136 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:24.619136 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.619136 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.619136 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:24.619136 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.619136 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:24.621316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:24.622054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:24.622054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.622054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:24.622054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.622054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.622054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:24.622054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.622054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:24.622914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.623696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.624598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:24.624598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.624598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:24.708403 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:24.708800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:24.709764 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:24.710203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.711122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:24.711122 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.711122 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:24.711122 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.711122 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.711122 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:24.711122 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.711122 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:24.712489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:24.713174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:24.713174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.713174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:24.713174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.713174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.713174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:24.713174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.713174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:24.714235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.714944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:24.715745 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.715745 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:24.756593 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:24.756857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:24.757382 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:24.757726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:24.758039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.758716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:24.758716 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.758716 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:24.758716 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.758716 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.758716 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:24.758716 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.758716 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.759851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:24.760700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:29.420622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:29.422494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:29.441298 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=252 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:22:29.468792 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:29.469433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.470656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:29.470656 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.470656 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:29.470656 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.470656 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.470656 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:29.470656 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.470656 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.472023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.472544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:29.584448 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:29.586101 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:29.586417 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:29.586593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.587345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:22:29.587345 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.587345 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:29.587345 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.587345 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.587345 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:29.587345 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.587345 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:29.588848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.589468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:29.617054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:29.617637 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:29.618371 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:29.618693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.619441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:29.619441 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.619441 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:29.619441 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.619441 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.619441 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:29.619441 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.619441 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:29.620200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.621200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:29.622176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:29.623456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:29.623456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:29.623456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:29.623456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:29.623456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.623456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:29.623456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.623456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.623456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:29.623456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.623456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:29.709058 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:29.709673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:29.710272 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:29.710724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.711494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:29.711494 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.711494 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:29.711494 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.711494 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.711494 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:29.711494 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.711494 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:29.712851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:29.713904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:29.713904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.713904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:29.713904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.713904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.713904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:29.713904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.713904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:29.714611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.715459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:29.716063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.716063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:29.757062 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:29.757587 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:29.758431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:29.759202 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:29.759459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.760591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:29.760591 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.760591 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:29.760591 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.760591 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.760591 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:29.760591 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.760591 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.762174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:29.763039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:34.421566 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:34.423426 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:34.456243 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:34.456664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.457508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:34.457508 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.457508 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:34.457508 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.457508 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.457508 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:34.457508 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.457508 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.458473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.459368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:34.585160 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:34.586418 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:34.586895 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:34.587147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.588100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:22:34.588100 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.588100 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:34.588100 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.588100 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.588100 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:34.588100 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.588100 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:34.589985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.590575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:34.617586 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:34.618012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:34.618895 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:34.619197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.620029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:34.620029 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.620029 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:34.620029 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.620029 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.620029 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:34.620029 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.620029 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:34.621290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:34.622426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:34.622426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.622426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:34.622426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.622426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.622426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:34.622426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.622426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:34.622823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.623565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.624376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:34.624376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.624376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:34.709589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:34.710047 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:34.710600 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:34.711041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.711832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:34.711832 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.711832 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:34.711832 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.711832 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.711832 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:34.711832 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.711832 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:34.713199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:34.713946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:34.713946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.713946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:34.713946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.713946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.713946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:34.713946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.713946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:34.715082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.715751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:34.716257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.716257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:34.757697 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:34.758090 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:34.758597 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:34.758935 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:34.759153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.760061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:34.760061 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.760061 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:34.760061 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.760061 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.760061 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:34.760061 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.760061 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.762958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:34.763979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:39.355440 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:39.364246 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:389}}
Mar 23 22:22:39.364640 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, security_groups=1, routers=1, router_ports=1 {{(pid=104232) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:370}}
Mar 23 22:22:39.364834 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Fixing resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:22:39.392277 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, resource={'id': '06cffee1-b8de-433e-a67a-ec0e950c9238', 'name': 'private', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['130adef4-306b-4397-a096-91ee5094d0e2', 'fb1912da-e258-4d41-8cd4-7896f0de6385'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:17:25Z', 'updated_at': '2026-03-23T22:17:28Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 36968}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.392277 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.392624 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c4286b-d53b-4217-a9f3-05177afa9297, external_ids_update=None, columns={'tag': 36968}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.392624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.392624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:39.392624 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.392624 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:39.392624 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.392624 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.392624 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:39.392624 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.392624 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:39.393751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Failed to fix resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.393751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:22:39.393751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:22:39.393751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:22:39.393751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:22:39.393751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:22:39.393751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:22:39.393751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:39.393751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.393751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.393751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.394412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.395141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:22:39.395141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.395141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:22:39.395141 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Fixing resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:22:39.422778 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:39.424601 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:39.428160 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, resource={'id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'name': 'public', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'a53179c9-417a-4560-9bc7-e3f64551b15b'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-23T22:17:31Z', 'updated_at': '2026-03-23T22:17:40Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.428160 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.428505 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a8bd993e-d23e-4e73-8066-766307763bdb, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.428505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.429467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:39.429467 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.429467 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:39.429467 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.429467 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.429467 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:39.429467 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.429467 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:39.430840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Failed to fix resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.430840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:22:39.430840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:22:39.430840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:22:39.430840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:22:39.430840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:22:39.430840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:22:39.430840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:39.430840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.430840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.430840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.431189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.431580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:22:39.431580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.431580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:22:39.431580 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Fixing resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:22:39.489390 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.491055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.491401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:22:39.491401 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.491401 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:39.491401 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.491401 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.491401 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:39.491401 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.491401 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.492656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.493440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:39.499449 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, resource={'id': '3a042d54-7a79-465e-bd57-47cd6833a4d0', 'name': 'admin_net', 'tenant_id': 'b4ffb2af173e4612a30978ef510f4f44', 'project_id': 'b4ffb2af173e4612a30978ef510f4f44', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['935d20c1-7d6b-489a-8358-b5fb94945b7e'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:20:57Z', 'updated_at': '2026-03-23T22:20:58Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 47875}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.499864 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.500641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c442d0-efff-4c7e-bcb6-377b4da39c34, external_ids_update=None, columns={'tag': 47875}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.500641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.501172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:39.501172 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.501172 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:39.501172 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.501172 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.501172 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:39.501172 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.501172 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:39.502331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Failed to fix resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.502331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:22:39.502331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:22:39.502331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:22:39.502331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:22:39.502331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:22:39.502331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:22:39.502331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:39.502331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.502331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.502331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.502878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.503263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:22:39.503263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.503263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:22:39.503263 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Fixing resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:22:39.537875 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, resource={'id': 'c9e6c30a-b30e-473b-b06a-50be75c90b36', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['421749b7-bb0d-4854-b3f4-f0414f4c37fb'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:21:34Z', 'updated_at': '2026-03-23T22:21:35Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 48210}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.538298 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.538827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13535482-a705-4453-af87-e8a7d8c10c82, external_ids_update=None, columns={'tag': 48210}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.539009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.539930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:39.539930 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.539930 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:39.539930 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.539930 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.539930 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:39.539930 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.539930 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:39.541052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Failed to fix resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.541052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:22:39.541052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:22:39.541052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:22:39.541052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:22:39.541052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:22:39.541052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:22:39.541052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:39.541052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.541052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.541052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.541572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.542144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:22:39.542144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.542144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:22:39.542144 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Fixing resource 183ad67d-09c0-4a7c-bf4a-75e3b69efd22 (type: security_groups) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:22:39.587614 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:39.589573 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.589787 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.589944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.590789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:22:39.590789 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.590789 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:39.590789 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.590789 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.590789 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:39.590789 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.590789 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:39.596123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.596628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:39.608568 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.db.ovn_revision_numbers_db [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Successfully bumped revision number for resource 183ad67d-09c0-4a7c-bf4a-75e3b69efd22 (type: security_groups) to 9
Mar 23 22:22:39.608657 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Fixing resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:22:39.621375 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:39.622864 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:39.624456 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.624864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.625753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:39.625753 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.625753 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:39.625753 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.625753 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.625753 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:39.625753 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.625753 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:39.628348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.629099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:39.629099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.629099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:39.629099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.629099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.629099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:39.629099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.629099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:39.630466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.631249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.631966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:39.631966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.631966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:39.710671 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:39.711827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:39.712564 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.715080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:39.715080 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.715080 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:39.715080 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.715080 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.715080 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:39.715080 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.715080 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:39.716084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.716851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:39.716851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.716851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:39.716851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.716851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.716851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:39.716851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.716851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:39.719174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.719877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:39.720428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.720428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:39.725443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, resource={'id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'enable_snat': True}, 'gw_port_id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-t62km', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'ip_address': '172.24.5.146', 'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a'}, {'ip_address': '2001:db8::396', 'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 3, 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d'}, resource_type=routers, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.725815 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.726434 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.726945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.728238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:39.728238 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.728238 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:39.728238 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.728238 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.728238 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:39.728238 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.728238 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:39.728585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Unable to update router 154c08d6-78c5-49fd-a785-0df3f4277a23. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.731572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Failed to fix resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.731572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:22:39.731572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:22:39.731572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:22:39.731572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:22:39.731572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:22:39.731572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 23 22:22:39.731572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 23 22:22:39.731572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 23 22:22:39.731572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 23 22:22:39.731572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:22:39.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:22:39.732781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.732781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:22:39.732781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.732781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:22:39.732781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.732781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.732781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:22:39.732781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.732781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:22:39.733769 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Fixing resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:22:39.759698 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:39.763881 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:39.766081 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.766462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.766957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.770914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:39.770914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.770914 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:39.770914 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.770914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.770914 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:39.770914 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.770914 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.772033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.772517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:39.893915 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, resource={'id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'name': '', 'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'project_id': '', 'mac_address': 'fa:16:3e:aa:0e:57', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160382', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.894387 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a a53179c9-417a-4560-9bc7-e3f64551b15b', 'neutron:network_name': 'neutron-bb9d5c47-4694-42c2-a018-18e3c9058915', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-154c08d6-78c5-49fd-a785-0df3f4277a23'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::396/64', '172.24.5.146/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.894895 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=348aba7c-88a8-4a19-9486-df3f7edb4007, lrouter_port=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:39.895363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.896533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Traceback (most recent call last):
Mar 23 22:22:39.896533 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.896533 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:39.896533 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.896533 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.896533 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:39.896533 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.896533 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:39.897898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Failed to fix resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.897898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:22:39.897898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:22:39.897898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:22:39.897898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:22:39.897898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:22:39.897898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 23 22:22:39.897898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:39.897898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.897898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.897898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:22:39.898481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:39.899088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:22:39.899088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:39.899088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:22:39.899088 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-58d5c95e-9d0f-411f-8aa6-6e6254d90ee6 None None] Maintenance task: Synchronization completed (took 0.53 seconds)
Mar 23 22:22:44.423545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:44.425351 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:44.457310 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:44.457678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.458481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:22:44.458481 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.458481 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:44.458481 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.458481 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.458481 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:44.458481 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.458481 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.459516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.460429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:44.589167 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:44.590410 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:44.590759 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:44.591009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.591907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:44.591907 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.591907 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:44.591907 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.591907 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.591907 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:44.591907 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.591907 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.593694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.594246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:44.622691 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:44.623236 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:44.623819 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:44.624100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.624908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:44.624908 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.624908 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:44.624908 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.624908 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.624908 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:44.624908 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.624908 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:44.626263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:44.627061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:44.627061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.627061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:44.627061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.627061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.627061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:44.627061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.627061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:44.627753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.628541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.629366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:44.629366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.629366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:44.711927 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:44.712376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:44.712844 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:44.713230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.713993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:44.713993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.713993 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:44.713993 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.713993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.713993 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:44.713993 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.713993 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:44.715249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:44.716008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:44.716008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.716008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:44.716008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.716008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.716008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:44.716008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.716008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:44.717030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.717835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:44.718769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.718769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:44.763849 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:44.764432 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:44.764911 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:44.765382 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:44.765597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.766430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:44.766430 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.766430 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:44.766430 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.766430 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.766430 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:44.766430 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.766430 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.768017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:44.768840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:49.424103 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:49.425855 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:49.457627 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:49.458031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.458962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:22:49.458962 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.458962 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:49.458962 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.458962 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.458962 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:49.458962 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.458962 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.459863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.460614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:49.589713 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:49.591163 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:49.591632 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:49.591893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.592993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:49.592993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.592993 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:49.592993 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.592993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.592993 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:49.592993 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.592993 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.595471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:49.623238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:49.623687 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:49.624306 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:49.624618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.625514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:49.625514 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.625514 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:49.625514 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.625514 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.625514 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:49.625514 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.625514 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:49.627080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:49.627542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:49.627542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.627542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:49.627542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.627542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.627542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:49.627542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.627542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:49.629492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.629917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.630397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:49.630397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.630397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:49.713106 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:49.713106 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:49.714971 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:49.714971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.714971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:49.714971 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.714971 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:49.714971 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.714971 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.714971 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:49.714971 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.714971 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:49.717085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.718194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:49.719327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:49.721072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:49.721072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:49.721072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.721072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:49.721072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.721072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.721072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:49.721072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.721072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:49.764350 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:49.764780 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:49.765365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:49.765821 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:49.766124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.766858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:49.766858 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.766858 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:49.766858 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.766858 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.766858 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:49.766858 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.766858 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.768785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:49.769645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:54.424831 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:54.426548 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:54.457671 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:54.459739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.459869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:22:54.459869 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.459869 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:54.459869 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.459869 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.459869 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:54.459869 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.459869 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.461521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.462770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:54.590523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:54.593146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:54.593146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:54.593146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.593488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:54.593488 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.593488 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:54.593488 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.593488 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.593488 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:54.593488 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.593488 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.595423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.595950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:54.623748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:54.624384 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:54.625981 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:54.625981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.627050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:54.627050 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.627050 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:54.627050 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.627050 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.627050 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:54.627050 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.627050 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:54.627714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:54.628772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:54.628772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.628772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:54.628772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.628772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.628772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:54.628772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.628772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:54.630657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.631476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.632345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:54.632345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.632345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:54.713295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:54.713697 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:54.714397 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:54.714946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.715717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:54.715717 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.715717 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:54.715717 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.715717 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.715717 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:54.715717 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.715717 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:54.716632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:54.717366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:54.717366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.717366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:54.717366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.717366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.717366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:54.717366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.717366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:54.717692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.718429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:54.719283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.719283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:54.764929 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:54.765448 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:54.765924 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:54.766326 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:54.766574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.767399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:54.767399 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.767399 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:54.767399 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.767399 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.767399 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:54.767399 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.767399 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.768973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:54.769863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:59.425820 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:59.427851 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:59.464777 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:59.465197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.465989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:22:59.465989 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.465989 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:59.465989 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.465989 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.465989 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:59.465989 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.465989 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.466882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.467652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:59.591495 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:59.593187 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:59.593615 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:59.593920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.595021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:59.595021 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.595021 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:59.595021 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.595021 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.595021 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:59.595021 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.595021 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.597152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.597520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:59.624396 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:59.624813 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:22:59.625443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:59.625767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.626560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:59.626560 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.626560 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:59.626560 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.626560 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.626560 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:59.626560 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.626560 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:59.627791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:59.628115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:59.628115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.628115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:59.628115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.628115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.628115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:59.628115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.628115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:22:59.629386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.629705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.630068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:59.630068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.630068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:59.713764 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:59.714549 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:59.715151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:59.715542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.716333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:59.716333 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.716333 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:59.716333 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.716333 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.716333 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:59.716333 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.716333 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:22:59.717642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:59.717988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:22:59.717988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.717988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:22:59.717988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.717988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.717988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:22:59.717988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.717988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:22:59.719663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.719985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:59.720353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.720353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:22:59.765399 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:22:59.766129 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:22:59.766850 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:59.767188 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:22:59.767446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.768152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:22:59.768152 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.768152 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:22:59.768152 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.768152 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.768152 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:22:59.768152 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.768152 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.769898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:22:59.770272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:04.438712 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:04.440500 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:04.476189 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:04.476519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.477256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:04.477256 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.477256 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:04.477256 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.477256 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.477256 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:04.477256 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.477256 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.478512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.479519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:04.592186 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:04.593775 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:04.593982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:04.594110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.594979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:04.594979 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.594979 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:04.594979 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.594979 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.594979 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:04.594979 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.594979 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.596926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.597726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:04.624924 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:04.625405 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:04.626003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:04.626418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.627243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:04.627243 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.627243 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:04.627243 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.627243 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.627243 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:04.627243 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.627243 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:04.628514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:04.629454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:04.629454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.629454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:04.629454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.629454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.629454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:04.629454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.629454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:04.630176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.631006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.631887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:04.631887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.631887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:04.714483 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:04.714749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:04.715371 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:04.715856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.716621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:04.716621 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.716621 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:04.716621 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.716621 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.716621 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:04.716621 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.716621 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:04.717381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:04.718073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:04.718073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.718073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:04.718073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.718073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.718073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:04.718073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.718073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:04.718524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.719244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:04.720110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.720110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:04.766033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:04.766619 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:04.767005 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:04.767451 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:04.767691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.768363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:04.768363 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.768363 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:04.768363 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.768363 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.768363 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:04.768363 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.768363 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.770582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:04.771499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:09.439778 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:09.482472 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:09.486447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:09.487741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.488660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:09.592983 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:09.597501 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:09.597803 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:09.597978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.599874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:09.599874 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.599874 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:09.599874 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.599874 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.599874 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:09.599874 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.599874 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.601448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.602429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:09.625500 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:09.625871 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:09.626852 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:09.627315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.628207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:09.628207 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.628207 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:09.628207 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.628207 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.628207 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:09.628207 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.628207 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:09.629734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:09.630606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:09.630606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.630606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:09.630606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.630606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.630606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:09.630606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.630606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:09.631548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.632015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.632584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:09.632584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.632584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:09.715116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:09.715793 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:09.716421 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:09.716874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.717775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:09.717775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.717775 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:09.717775 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.717775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.717775 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:09.717775 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.717775 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:09.719035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:09.719770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:09.719770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.719770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:09.719770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.719770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.719770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:09.719770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.719770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:09.720682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.721416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:09.722168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.722168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:09.766801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:09.767554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:09.768685 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:09.768941 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:09.769097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.769790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:09.769790 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.769790 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:09.769790 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.769790 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.769790 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:09.769790 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.769790 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.771244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:09.772400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:14.441570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:14.442501 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:14.482967 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:14.483408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.484424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:14.484424 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.484424 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:14.484424 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.484424 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.484424 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:14.484424 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.484424 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.485436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.486046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:14.596851 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:14.597882 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:14.598193 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:14.598497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.599309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:14.599309 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.599309 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:14.599309 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.599309 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.599309 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:14.599309 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.599309 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.601105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.601856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:14.626344 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:14.626890 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:14.627363 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:14.627648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.628362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:14.628362 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.628362 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:14.628362 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.628362 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.628362 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:14.628362 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.628362 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:14.629481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:14.630271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:14.630271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.630271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:14.630271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.630271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.630271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:14.630271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.630271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:14.630977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.631724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.632601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:14.632601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.632601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:14.715890 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:14.716234 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:14.717239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:14.717823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.718690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:14.718690 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.718690 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:14.718690 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.718690 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.718690 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:14.718690 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.718690 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:14.719856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:14.720691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:14.720691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.720691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:14.720691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.720691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.720691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:14.720691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.720691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:14.721261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.721955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:14.722902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.722902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:14.767365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:14.767917 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:14.768586 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:14.768925 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:14.769167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.770019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:14.770019 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.770019 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:14.770019 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.770019 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.770019 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:14.770019 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.770019 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.771811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:14.772677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:19.441707 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:19.443266 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:19.477201 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:19.477768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.478855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:19.478855 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.478855 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:19.478855 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.478855 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.478855 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:19.478855 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.478855 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.480066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.480899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:19.597760 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:19.599462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:19.599882 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:19.600125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.601322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:19.601322 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.601322 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:19.601322 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.601322 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.601322 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:19.601322 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.601322 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.603024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.603857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:19.626954 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:19.627377 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:19.627958 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:19.628094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.628644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:19.628644 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.628644 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:19.628644 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.628644 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.628644 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:19.628644 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.628644 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:19.629469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.630675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:19.631466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:19.632434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:19.632434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:19.632434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:19.632434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:19.632434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.632434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:19.632434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.632434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.632434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:19.632434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.632434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:19.716462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:19.716927 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:19.717911 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:19.718354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.719231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:19.719231 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.719231 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:19.719231 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.719231 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.719231 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:19.719231 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.719231 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:19.720686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:19.721573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:19.721573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.721573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:19.721573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.721573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.721573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:19.721573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.721573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:19.722271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.723013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:19.723856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.723856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:19.767841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:19.768549 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:19.769141 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:19.769410 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:19.769596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.770318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:19.770318 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.770318 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:19.770318 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.770318 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.770318 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:19.770318 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.770318 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.772109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:19.772892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:24.442382 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:24.444151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:24.483707 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:24.484446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.484446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:24.484446 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.484446 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:24.484446 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.484446 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.484446 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:24.484446 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.484446 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.485567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.485929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:24.598364 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:24.599480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:24.599900 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:24.600161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.600995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:24.600995 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.600995 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:24.600995 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.600995 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.600995 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:24.600995 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.600995 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.602349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.603054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:24.627519 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:24.627881 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:24.628675 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:24.628876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.629730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:24.629730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.629730 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:24.629730 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.629730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.629730 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:24.629730 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.629730 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:24.630920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:24.631635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:24.631635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.631635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:24.631635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.631635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.631635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:24.631635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.631635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:24.632717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.633388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.634203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:24.634203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.634203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:24.717138 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:24.717787 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:24.718745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:24.719194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.720128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:24.720128 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.720128 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:24.720128 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.720128 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.720128 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:24.720128 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.720128 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:24.721550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:24.722374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:24.722374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.722374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:24.722374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.722374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.722374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:24.722374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.722374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:24.723242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.723907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:24.724708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.724708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:24.768388 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:24.769071 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:24.769791 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:24.770269 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:24.770531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.771431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:24.771431 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.771431 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:24.771431 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.771431 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.771431 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:24.771431 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.771431 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:24.773957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:29.443205 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:29.444971 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:29.452323 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=450,neutron:UPDATE=1 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:23:29.479164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:29.479556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.480437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:29.480437 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.480437 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:29.480437 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.480437 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.480437 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:29.480437 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.480437 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.481985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.482769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:29.599322 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:29.601344 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:29.601740 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:29.602053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.603013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:29.603013 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.603013 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:29.603013 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.603013 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.603013 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:29.603013 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.603013 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.604696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.605823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:29.628900 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:29.628900 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:29.629678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:29.630013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.630952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:29.630952 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.630952 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:29.630952 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.630952 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.630952 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:29.630952 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.630952 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:29.631798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:29.632702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:29.632702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.632702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:29.632702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.632702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.632702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:29.632702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.632702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:29.633138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.634013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.634986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:29.634986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.634986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:29.717944 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:29.718295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:29.718958 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:29.719378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.720109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:29.720109 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.720109 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:29.720109 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.720109 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.720109 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:29.720109 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.720109 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:29.721239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:29.722021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:29.722021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.722021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:29.722021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.722021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.722021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:29.722021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.722021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:29.722439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.723388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:29.724338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.724338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:29.768947 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:29.769711 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:29.770339 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:29.770713 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:29.770946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.771826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:29.771826 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.771826 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:29.771826 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.771826 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.771826 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:29.771826 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.771826 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.773682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:29.774619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:34.444059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:34.445196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:34.482168 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:34.482350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.483346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:34.483346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.483346 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:34.483346 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.483346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.483346 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:34.483346 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.483346 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.483778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.484315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:34.600812 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:34.601570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:34.601946 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:34.602201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.603118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:34.603118 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.603118 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:34.603118 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.603118 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.603118 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:34.603118 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.603118 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.604881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.605822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:34.628838 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:34.629425 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:34.629996 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:34.630354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.631144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:34.631144 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.631144 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:34.631144 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.631144 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.631144 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:34.631144 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.631144 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:34.632517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:34.632946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:34.632946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.632946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:34.632946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.632946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.632946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:34.632946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.632946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:34.634137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.634574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.635038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:34.635038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.635038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:34.718505 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:34.718946 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:34.719628 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:34.720009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.720778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:34.720778 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.720778 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:34.720778 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.720778 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.720778 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:34.720778 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.720778 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:34.722074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:34.722766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:34.722766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.722766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:34.722766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.722766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.722766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:34.722766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.722766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:34.723728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.724257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:34.724754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.724754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:34.769505 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:34.769912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:34.770486 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:34.770841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:34.771065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.771812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:34.771812 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.771812 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:34.771812 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.771812 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.771812 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:34.771812 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.771812 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.773573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:34.774399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:39.444709 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:39.446251 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:39.486789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:39.487246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.488394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:39.488394 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.488394 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:39.488394 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.488394 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.488394 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:39.488394 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.488394 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.490499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.491354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:39.600963 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:39.602357 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:39.602647 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:39.602812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.603608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:39.603608 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.603608 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:39.603608 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.603608 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.603608 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:39.603608 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.603608 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.605202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.606085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:39.629493 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:39.633480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:39.633983 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:39.634239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.634854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:39.634854 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.634854 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:39.634854 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.634854 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.634854 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:39.634854 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.634854 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:39.636485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:39.637260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:39.637260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.637260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:39.637260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.637260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.637260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:39.637260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.637260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:39.638274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.639516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.640495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:39.640495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.640495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:39.719071 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:39.719443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:39.720015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:39.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.721817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:39.721817 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.721817 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:39.721817 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.721817 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.721817 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:39.721817 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.721817 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:39.722320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.723099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:39.723831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:39.724708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:39.724708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:39.724708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.724708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:39.724708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.724708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.724708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:39.724708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.724708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:39.769869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:39.770094 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:39.770393 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:39.770611 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:39.770713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.771029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:39.771029 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.771029 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:39.771029 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.771029 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.771029 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:39.771029 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.771029 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.771697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:39.772521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:44.445459 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:44.447091 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:44.480094 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:44.480542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.481762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:44.481762 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.481762 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:44.481762 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.481762 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.481762 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:44.481762 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.481762 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.482996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.483862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:44.601744 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:44.602941 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:44.603307 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:44.603578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.604509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:44.604509 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.604509 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:44.604509 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.604509 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.604509 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:44.604509 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.604509 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.605736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.606533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:44.633589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:44.633838 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:44.634336 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:44.634666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.635403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:44.635403 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.635403 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:44.635403 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.635403 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.635403 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:44.635403 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.635403 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:44.636502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:44.637192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:44.637192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.637192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:44.637192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.637192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.637192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:44.637192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.637192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:44.637685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.638418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.639203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:44.639203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.639203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:44.719619 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:44.720012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:44.720498 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:44.720842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.721520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:44.721520 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.721520 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:44.721520 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.721520 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.721520 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:44.721520 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.721520 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:44.722673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:44.723417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:44.723417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.723417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:44.723417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.723417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.723417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:44.723417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.723417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:44.724524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.725255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:44.725739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.725739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:44.770400 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:44.770782 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:44.771123 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:44.771520 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:44.771731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.772402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:44.772402 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.772402 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:44.772402 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.772402 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.772402 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:44.772402 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.772402 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.773999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:44.774775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:49.446268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:49.449490 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:49.481449 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:49.481779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:49.484067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:49.484650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.485169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:49.485169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.485169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.485169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:49.485169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.485169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:49.602382 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:49.604079 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:49.604478 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:49.604731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.605601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:49.605601 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.605601 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:49.605601 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.605601 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.605601 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:49.605601 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.605601 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.606781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.607126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:49.634144 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:49.634623 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:49.635406 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:49.635406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.636323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:23:49.636323 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.636323 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:49.636323 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.636323 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.636323 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:49.636323 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.636323 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:49.637589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:49.638338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:49.638338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:49.638338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.638338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:49.638338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.638338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.638338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:49.638338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.638338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:49.640535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.641036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.641575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:49.641575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.641575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:49.720150 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:49.720690 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:49.721306 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:49.721680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:49.722507 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.722507 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:49.722507 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.722507 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.722507 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:49.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:49.723847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:49.724576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:49.724576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.724576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:49.724576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.724576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.724576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:49.724576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.724576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:49.725195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.725738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:49.726273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.726273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:49.770847 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:49.771514 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:49.771880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:49.772149 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:49.772311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.772904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:49.772904 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.772904 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:49.772904 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.772904 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.772904 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:49.772904 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.772904 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.774569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:49.775564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:54.447791 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:54.449648 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:54.480257 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:54.480700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.481641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:54.481641 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.481641 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:54.481641 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.481641 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.481641 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:54.481641 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.481641 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.482557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.483481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:54.602972 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:54.604648 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:54.604971 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:54.605250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.606243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:54.606243 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.606243 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:54.606243 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.606243 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.606243 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:54.606243 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.606243 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.608079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.608879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:54.634535 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:54.634924 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:54.635475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:54.635778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.636618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:54.636618 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.636618 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:54.636618 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.636618 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.636618 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:54.636618 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.636618 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:54.637813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:54.638620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:54.638620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.638620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:54.638620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.638620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.638620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:54.638620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.638620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:54.639397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.640110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.640950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:54.640950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.640950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:54.720802 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:54.721353 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:54.722114 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:54.722345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.723115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:54.723115 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.723115 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:54.723115 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.723115 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.723115 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:54.723115 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.723115 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:54.724558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:54.725334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:54.725334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.725334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:54.725334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.725334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.725334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:54.725334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.725334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:54.727443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.727945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:54.729242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.729242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:54.771414 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:54.771928 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:54.772477 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:54.772856 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:54.773085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.773844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:54.773844 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.773844 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:54.773844 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.773844 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.773844 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:54.773844 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.773844 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.775594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:54.776439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:59.448630 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:59.450583 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:59.486513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:59.486906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.487898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:23:59.487898 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.487898 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:59.487898 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.487898 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.487898 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:59.487898 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.487898 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.489113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.490267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:59.603650 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:59.604858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:59.605179 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:59.605466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.606341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:59.606341 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.606341 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:59.606341 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.606341 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.606341 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:59.606341 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.606341 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.608121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.608974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:59.634927 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:59.635330 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:23:59.635733 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:59.635907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.636401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:59.636401 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.636401 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:59.636401 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.636401 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.636401 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:59.636401 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.636401 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:59.637288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:59.637992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:59.637992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.637992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:59.637992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.637992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.637992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:59.637992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.637992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:23:59.638400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.639067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.639893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:59.639893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.639893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:59.721315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:59.729044 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:59.729858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:59.730108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.730712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:59.730712 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.730712 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:59.730712 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.730712 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.730712 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:59.730712 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.730712 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:23:59.731394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:59.732077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:23:59.732077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.732077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:23:59.732077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.732077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.732077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:23:59.732077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.732077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:23:59.732599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.733297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:59.734083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.734083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:23:59.771871 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:23:59.772084 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:23:59.772422 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:59.772642 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:23:59.772795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.773248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:23:59.773248 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.773248 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:23:59.773248 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.773248 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.773248 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:23:59.773248 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.773248 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.774088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:23:59.774856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:04.449608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:04.451672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:04.487668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:04.488067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.488998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:04.488998 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.488998 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:04.488998 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.488998 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.488998 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:04.488998 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.488998 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.490811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:04.604551 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:04.606159 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:04.606585 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:04.606812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.607913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:04.607913 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.607913 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:04.607913 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.607913 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.607913 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:04.607913 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.607913 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.610107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.610948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:04.635498 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:04.636117 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:04.636718 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:04.636958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.637883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:04.637883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.637883 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:04.637883 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.637883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.637883 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:04.637883 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.637883 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:04.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:04.639988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:04.639988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.639988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:04.639988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.639988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.639988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:04.639988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.639988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:04.640797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.641533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.642491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:04.642491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.642491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:04.729535 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:04.729912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:04.730816 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:04.731201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.732107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:04.732107 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.732107 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:04.732107 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.732107 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.732107 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:04.732107 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.732107 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:04.733459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:04.734174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:04.734174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.734174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:04.734174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.734174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.734174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:04.734174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.734174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:04.735306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.736024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:04.736874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.736874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:04.789574 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:04.790324 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:04.790599 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:04.790783 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:04.790995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.791836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:04.791836 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.791836 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:04.791836 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.791836 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.791836 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:04.791836 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.791836 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.792287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:04.793072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:09.450569 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:09.452377 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:09.489606 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:09.489800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.490437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:09.490437 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.490437 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:09.490437 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.490437 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.490437 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:09.490437 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.490437 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.491696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.492702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:09.605316 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:09.606668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:09.607442 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:09.607442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.608494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:09.608494 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.608494 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:09.608494 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.608494 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.608494 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:09.608494 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.608494 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.610504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.611030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:09.635948 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:09.636457 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:09.637151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:09.637554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.638450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:09.638450 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.638450 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:09.638450 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.638450 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.638450 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:09.638450 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.638450 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:09.639738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:09.640610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:09.640610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.640610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:09.640610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.640610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.640610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:09.640610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.640610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:09.641640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.642385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.643174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:09.643174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.643174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:09.730145 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:09.731116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:09.731778 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:09.732163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:09.736176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:09.737959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.738706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:09.790520 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:09.790930 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:09.792428 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:09.792780 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:09.793045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.822465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:09.822465 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.822465 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:09.822465 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.822465 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.822465 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:09.822465 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.822465 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.824510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:09.825464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:14.451202 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:14.452358 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:14.495165 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:14.495165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.495622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:14.495622 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.495622 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:14.495622 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.495622 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.495622 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:14.495622 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.495622 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.496563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.497112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:14.607768 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:14.609340 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:14.609527 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:14.609662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.610287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:14.610287 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.610287 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:14.610287 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.610287 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.610287 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:14.610287 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.610287 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.611416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.612235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:14.636427 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:14.636805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:14.637430 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:14.637626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.638073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:14.638073 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.638073 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:14.638073 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.638073 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.638073 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:14.638073 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.638073 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:14.638891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:14.639451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:14.639451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.639451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:14.639451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.639451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.639451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:14.639451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.639451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:14.640476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.640965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.641531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:14.641531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.641531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:14.731022 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:14.733454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:14.735159 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:14.735399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.735967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:14.735967 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.735967 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:14.735967 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.735967 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.735967 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:14.735967 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.735967 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:14.736628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:14.737087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:14.737087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.737087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:14.737087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.737087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.737087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:14.737087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.737087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:14.737368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.737824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:14.738358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.738358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:14.791003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:14.791489 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:14.791806 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:14.792015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:14.792138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.792786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:14.792786 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.792786 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:14.792786 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.792786 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.792786 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:14.792786 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.792786 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.793674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:14.794152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:19.451951 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:19.455342 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:19.491814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:19.492317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.493288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:19.493288 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.493288 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:19.493288 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.493288 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.493288 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:19.493288 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.493288 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.494743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.495627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:19.609050 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:19.610238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:19.610501 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:19.610622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.611327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:19.611327 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.611327 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:19.611327 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.611327 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.611327 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:19.611327 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.611327 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.612547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.613465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:19.640628 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:19.640970 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:19.641528 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:19.641528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.642134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:24:19.642134 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.642134 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:19.642134 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.642134 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.642134 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:19.642134 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.642134 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:19.643632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:19.644643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:19.644643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:19.644643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.644643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:19.644643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.644643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.644643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:19.644643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.644643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:19.646743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.647827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.648730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:19.648730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.648730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:19.733807 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:19.734190 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:19.734725 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:19.734987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.735597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:19.735597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.735597 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:19.735597 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.735597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.735597 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:19.735597 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.735597 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:19.736889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:19.737698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:19.737698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.737698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:19.737698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.737698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.737698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:19.737698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.737698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:19.738760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.739619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:19.740494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.740494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:19.791738 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:19.792148 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:19.792581 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:19.792847 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:19.792973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.793557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:19.793557 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.793557 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:19.793557 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.793557 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.793557 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:19.793557 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.793557 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.795488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:19.796293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:24.452746 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:24.454086 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:24.488560 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:24.489030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.490256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:24.490256 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.490256 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:24.490256 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.490256 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.490256 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:24.490256 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.490256 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.491448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.492262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:24.609751 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:24.611275 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:24.611709 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:24.611958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.613021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:24.613021 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.613021 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:24.613021 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.613021 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.613021 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:24.613021 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.613021 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.615037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.615647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:24.640770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:24.642150 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:24.643118 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:24.643484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.644257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:24:24.644257 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.644257 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:24.644257 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.644257 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.644257 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:24.644257 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.644257 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:24.645274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:24.645733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:24.645733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:24.645733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.645733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:24.645733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.645733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.645733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:24.645733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.645733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:24.646835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.647446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:24.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:24.734237 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:24.735510 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:24.736694 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:24.737194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.738362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:24.738362 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.738362 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:24.738362 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.738362 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.738362 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:24.738362 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.738362 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:24.739688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:24.740037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:24.740037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.740037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:24.740037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.740037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.740037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:24.740037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.740037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:24.742531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.743044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:24.743543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.743543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:24.792236 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:24.793345 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:24.793994 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:24.794413 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:24.794668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.795611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:24.795611 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.795611 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:24.795611 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.795611 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.795611 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:24.795611 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.795611 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.797474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:24.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:29.453170 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:29.453897 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:29.485687 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:24:29.528686 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:29.529056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.529808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:29.529808 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.529808 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:29.529808 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.529808 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.529808 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:29.529808 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.529808 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.530600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.530965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.645729 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.646492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.647206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Traceback (most recent call last):
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.648265 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:29.648863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:29.649705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:29.650344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.651680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:29.735780 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:29.736334 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:29.737057 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:29.737495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.738365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:29.738365 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.738365 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:29.738365 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.738365 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.738365 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:29.738365 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.738365 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:29.739629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:29.740367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:29.740367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.740367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:29.740367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.740367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.740367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:29.740367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.740367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:29.741558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.742263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:29.743028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.743028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:29.792818 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:29.794327 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:29.795428 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:29.796705 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:29.796705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.797733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:29.797733 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.797733 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:29.797733 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.797733 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.797733 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:29.797733 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.797733 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.800006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:29.800613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:34.453708 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:34.454738 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:34.494336 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:34.494611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.495322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:34.495322 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.495322 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:34.495322 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.495322 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.495322 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:34.495322 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.495322 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.496506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.497324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:34.611193 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:34.612405 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:34.612680 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:34.612954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.613588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:34.613588 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.613588 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:34.613588 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.613588 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.613588 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:34.613588 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.613588 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.615779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.616616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:34.642172 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:34.643088 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:34.644285 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:34.645272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.645977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:24:34.645977 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.645977 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:34.645977 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.645977 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.645977 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:34.645977 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.645977 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:34.648527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:34.649198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:34.649198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:34.649198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.649198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:34.649198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.649198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.649198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:34.649198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.649198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:34.649869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.650353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.650909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:34.650909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.650909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:34.736445 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:34.737228 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:34.738081 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:34.738429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.739031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:34.739031 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.739031 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:34.739031 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.739031 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.739031 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:34.739031 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.739031 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:34.740481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:34.741151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:34.741151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.741151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:34.741151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.741151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.741151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:34.741151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.741151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:34.742266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.743064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:34.743907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.743907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:34.794177 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:34.794714 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:34.795419 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:34.795636 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:34.795766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.796500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:34.796500 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.796500 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:34.796500 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.796500 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.796500 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:34.796500 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.796500 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.798470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:34.799613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:39.454274 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:39.455373 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:39.486431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:39.486705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.487379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:39.487379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.487379 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:39.487379 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.487379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.487379 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:39.487379 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.487379 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.488289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.488714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:39.611845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:39.613133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:39.613554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:39.614011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.614697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:39.614697 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.614697 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:39.614697 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.614697 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.614697 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:39.614697 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.614697 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.616355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.617141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:39.642680 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:39.643480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:39.644180 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:39.644539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.646064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:24:39.646064 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.646064 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:39.646064 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.646064 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.646064 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:39.646064 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.646064 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:39.647477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:39.647910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:39.647910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:39.647910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.647910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:39.647910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.647910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.647910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:39.647910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.647910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:39.649403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.649826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.650288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:39.650288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.650288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:39.737278 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:39.737561 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:39.738342 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:39.739121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.740655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:39.740655 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.740655 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:39.740655 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.740655 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.740655 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:39.740655 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.740655 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:39.741891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:39.742654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:39.742654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.742654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:39.742654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.742654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.742654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:39.742654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.742654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:39.743822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.744289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:39.744797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.744797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:39.794712 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:39.795134 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:39.796204 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:39.796737 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:39.797005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.798123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:39.798123 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.798123 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:39.798123 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.798123 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.798123 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:39.798123 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.798123 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.799916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:39.800871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:44.455320 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:44.457668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:44.500197 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:44.500791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.501885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:44.501885 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.501885 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:44.501885 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.501885 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.501885 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:44.501885 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.501885 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.502966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.503377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:44.612718 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:44.614750 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:44.615084 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:44.615428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.616594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:44.616594 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.616594 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:44.616594 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.616594 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.616594 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:44.616594 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.616594 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.619182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.619886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:44.643392 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:44.643988 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f588b869-684c-4888-b165-2d1e72e8d3a5 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:44.644996 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:44.645626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.646399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f588b869-684c-4888-b165-2d1e72e8d3a5 None None] Traceback (most recent call last):
Mar 23 22:24:44.646399 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.646399 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:44.646399 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.646399 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.646399 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:44.646399 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.646399 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f588b869-684c-4888-b165-2d1e72e8d3a5 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:44.648151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:44.649815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:44.649815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:44.649815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.649815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:44.649815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.649815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.649815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:44.649815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.649815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-f588b869-684c-4888-b165-2d1e72e8d3a5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:44.650391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.651622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.653358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:44.653358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.653358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:44.737677 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:44.738127 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:44.738761 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:44.739245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.740013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:44.740013 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.740013 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:44.740013 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.740013 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.740013 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:44.740013 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.740013 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:44.740680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.741575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:44.742488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:44.743547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:44.743547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:44.743547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.743547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:44.743547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.743547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.743547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:44.743547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.743547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:44.795346 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:44.795729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:44.796310 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:44.796738 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:44.797034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.797843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:44.797843 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.797843 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:44.797843 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.797843 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.797843 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:44.797843 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.797843 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.799707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:44.801094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:49.455995 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:49.457815 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:49.493233 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:49.493694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.494643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:49.494643 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.494643 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:49.494643 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.494643 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.494643 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:49.494643 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.494643 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.495919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.496784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:49.613679 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:49.615287 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:49.615664 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:49.615953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.616859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:49.616859 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.616859 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:49.616859 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.616859 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.616859 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:49.616859 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.616859 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.618107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.619424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:49.643873 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:49.644127 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:49.644414 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:49.644576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.644972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:49.644972 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.644972 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:49.644972 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.644972 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.644972 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:49.644972 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.644972 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:49.645683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:49.646532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:49.646532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.646532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:49.646532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.646532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.646532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:49.646532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.646532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:49.646880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.647617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.648581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:49.648581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.648581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:49.739154 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:49.739978 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:49.740662 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:49.741025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:49.741752 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.741752 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:49.741752 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.741752 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.741752 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:49.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.741752 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:49.743065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:49.743399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:49.743399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.743399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:49.743399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.743399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.743399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:49.743399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.743399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:49.744895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.745261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:49.745587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.745587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:49.795985 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:49.796286 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:49.796903 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:49.797180 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:49.797371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.797964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:49.797964 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.797964 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:49.797964 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.797964 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.797964 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:49.797964 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.797964 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.799099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:49.799461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:54.456593 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:54.457427 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:54.489711 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:54.489924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.490547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:54.490547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.490547 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:54.490547 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.490547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.490547 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:54.490547 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.490547 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.491435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.492233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:54.614541 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:54.615789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:54.616070 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:54.616360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.617132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:54.617132 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.617132 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:54.617132 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.617132 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.617132 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:54.617132 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.617132 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.618933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.619757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:54.644315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:54.644700 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:54.645328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:54.645521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.646288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:54.646288 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.646288 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:54.646288 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.646288 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.646288 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:54.646288 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.646288 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:54.647181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:54.647657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:54.647657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.647657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:54.647657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.647657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.647657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:54.647657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.647657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:54.648241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.648635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.649118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:54.649118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.649118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:54.740151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:54.740668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:54.741439 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:54.741834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.742639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:54.742639 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.742639 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:54.742639 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.742639 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.742639 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:54.742639 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.742639 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:54.743647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:54.744086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:54.744086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.744086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:54.744086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.744086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.744086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:54.744086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.744086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:54.744702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.745161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:54.745707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.745707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:54.796640 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:54.797528 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:54.797952 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:54.798204 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:54.798389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.800385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:54.800385 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.800385 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:54.800385 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.800385 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.800385 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:54.800385 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.800385 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.800933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:54.801542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:59.457195 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:59.458668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:59.489805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:59.490198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.491096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:24:59.491096 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.491096 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:59.491096 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.491096 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.491096 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:59.491096 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.491096 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.492362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.493366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:59.615376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:59.616651 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:59.616944 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:59.617158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.618060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:59.618060 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.618060 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:59.618060 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.618060 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.618060 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:59.618060 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.618060 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.619816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.620668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:59.645021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:59.645411 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:24:59.645960 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:59.646167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.646708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:59.646708 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.646708 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:59.646708 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.646708 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.646708 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:59.646708 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.646708 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:59.647704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:59.648048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:59.648048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.648048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:59.648048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.648048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.648048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:59.648048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.648048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:24:59.648675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.649138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.649587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:59.649587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.649587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:59.740876 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:59.741778 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:59.741778 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:59.742082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.743658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:59.743658 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.743658 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:59.743658 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.743658 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.743658 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:59.743658 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.743658 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:24:59.745111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:59.745892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:24:59.745892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.745892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:24:59.745892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.745892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.745892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:24:59.745892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.745892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:24:59.746315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.747018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:59.747991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.747991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:24:59.797428 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:24:59.797887 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:24:59.798603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:59.799012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:24:59.799289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.800120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:24:59.800120 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.800120 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:24:59.800120 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.800120 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.800120 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:24:59.800120 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.800120 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.801860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:24:59.802628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:04.457916 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:04.460145 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:04.493548 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:04.493862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.494678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:04.494678 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.494678 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:04.494678 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.494678 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.494678 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:04.494678 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.494678 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.496040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.496860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:04.615810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:04.617062 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:04.617414 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:04.617800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.618587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:04.618587 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.618587 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:04.618587 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.618587 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.618587 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:04.618587 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.618587 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.620140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.621331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:04.645695 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:04.646073 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:04.646959 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:04.647236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.648128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:04.648128 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.648128 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:04.648128 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.648128 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.648128 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:04.648128 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.648128 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:04.648838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.649636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:04.650357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:04.651249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:04.651249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:04.651249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:04.651249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:04.651249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.651249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:04.651249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.651249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.651249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:04.651249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.651249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:04.741207 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:04.741774 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:04.742678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:04.743255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.744379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:04.744379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.744379 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:04.744379 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.744379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.744379 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:04.744379 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.744379 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:04.745679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:04.746717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:04.746717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.746717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:04.746717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.746717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.746717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:04.746717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.746717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:04.747321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.748406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:04.749829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.749829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:04.797961 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:04.798375 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:04.798908 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:04.799531 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:04.799804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.800607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:04.800607 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.800607 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:04.800607 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.800607 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.800607 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:04.800607 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.800607 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.802280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:04.803060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:09.458762 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:09.460386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:09.496897 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:09.497338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.498236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:09.498236 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.498236 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:09.498236 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.498236 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.498236 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:09.498236 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.498236 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.499554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:09.616517 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:09.617888 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:09.618146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:09.618408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.621255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:09.622450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.623612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.623612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:09.623612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.623612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:09.646245 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:09.646631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:09.647674 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:09.647674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.648203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:09.648203 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.648203 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:09.648203 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.648203 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.648203 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:09.648203 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.648203 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:09.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:09.650402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:09.650402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.650402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:09.650402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.650402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.650402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:09.650402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.650402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:09.651002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.652375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.653104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:09.653104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.653104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:09.741835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:09.742131 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:09.743007 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:09.743513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.744435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:09.744435 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.744435 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:09.744435 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.744435 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.744435 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:09.744435 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.744435 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:09.745265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:09.745975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:09.745975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.745975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:09.745975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.745975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.745975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:09.745975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.745975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:09.746318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.746751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:09.747347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.747347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:09.798413 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:09.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:09.799678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:09.799918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:09.800087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.800959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:09.800959 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.800959 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:09.800959 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.800959 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.800959 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:09.800959 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.800959 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.802479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:09.803365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:14.459430 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:14.460981 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:14.495676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:14.495976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.496840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:14.496840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.496840 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:14.496840 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.496840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.496840 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:14.496840 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.496840 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.497730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.498673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:14.617391 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:14.618802 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:14.619121 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:14.619460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.620719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:14.620719 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.620719 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:14.620719 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.620719 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.620719 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:14.620719 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.620719 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.622531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.623327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:14.646754 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:14.647099 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:14.647854 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:14.648179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.649038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:14.649038 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.649038 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:14.649038 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.649038 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.649038 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:14.649038 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.649038 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:14.650392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:14.651311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:14.651311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.651311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:14.651311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.651311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.651311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:14.651311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.651311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:14.652161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.653563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.654986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:14.654986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.654986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:14.742435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:14.742874 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:14.743562 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:14.743867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.744542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:14.744542 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.744542 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:14.744542 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.744542 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.744542 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:14.744542 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.744542 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:14.745862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:14.746611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:14.746611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.746611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:14.746611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.746611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.746611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:14.746611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.746611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:14.747593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.748359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:14.749170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.749170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:14.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:14.799626 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:14.800061 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:14.800352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:14.800517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.801103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:14.801103 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.801103 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:14.801103 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.801103 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.801103 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:14.801103 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.801103 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.802835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:14.803774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:19.460020 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:19.461576 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:19.491179 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:19.491644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.492490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:19.492490 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.492490 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:19.492490 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.492490 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.492490 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:19.492490 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.492490 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.493628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.494422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:19.618174 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:19.619170 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:19.619604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:19.619847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.620703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:19.620703 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.620703 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:19.620703 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.620703 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.620703 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:19.620703 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.620703 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.621931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.622555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:19.647200 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:19.647554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:19.647983 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:19.648301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.648969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:19.648969 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.648969 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:19.648969 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.648969 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.648969 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:19.648969 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.648969 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:19.650183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:19.650995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:19.650995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.650995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:19.650995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.650995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.650995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:19.650995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.650995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:19.651762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.652233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.652828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:19.652828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.652828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:19.742805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:19.742945 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:19.743641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:19.744021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.744752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:19.744752 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.744752 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:19.744752 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.744752 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.744752 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:19.744752 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.744752 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:19.745552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:19.746313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:19.746313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.746313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:19.746313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.746313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.746313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:19.746313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.746313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:19.746691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.747456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:19.748294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.748294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:19.803379 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:19.803956 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:19.804547 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:19.804918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:19.805168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.805911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:19.805911 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.805911 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:19.805911 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.805911 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.805911 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:19.805911 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.805911 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.807016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:19.807890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:24.460914 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:24.461934 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:24.495442 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:24.495865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.496775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:24.496775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.496775 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:24.496775 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.496775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.496775 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:24.496775 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.496775 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.497994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.498849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:24.618830 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:24.620209 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:24.620531 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:24.620881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.621879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:24.621879 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.621879 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:24.621879 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.621879 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.621879 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:24.621879 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.621879 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.623688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.624520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:24.647534 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:24.647994 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:24.648633 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:24.648906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.649724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:24.649724 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.649724 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:24.649724 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.649724 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.649724 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:24.649724 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.649724 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:24.651284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:24.652130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:24.652130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.652130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:24.652130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.652130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.652130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:24.652130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.652130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:24.653071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.653800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.654607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:24.654607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.654607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:24.743459 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:24.743702 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:24.744511 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:24.744995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.745909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:24.745909 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.745909 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:24.745909 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.745909 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.745909 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:24.745909 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.745909 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:24.746694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:24.747554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:24.747554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.747554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:24.747554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.747554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.747554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:24.747554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.747554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:24.748041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.748717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:24.749528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.749528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:24.803850 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:24.804860 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:24.805621 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:24.806048 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:24.806344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.807392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:24.807392 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.807392 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:24.807392 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.807392 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.807392 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:24.807392 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.807392 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.809349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:24.810829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:29.461772 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:29.463236 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:29.498052 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:29.498482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.499421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:29.499421 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.499421 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:29.499421 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.499421 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.499421 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:29.499421 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.499421 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.500379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.501128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:29.619847 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:29.620688 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:29.620990 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:29.621237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.622002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:29.622002 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.622002 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:29.622002 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.622002 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.622002 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:29.622002 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.622002 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.623911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.624975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.666302 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:29.667021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.667556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:29.668001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:29.668537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:29.668537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:29.668537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:29.668537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:29.668537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.668537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:29.668537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.668537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.668537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:29.668537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.668537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:29.743996 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:29.744650 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:29.745229 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:29.745691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.747183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:29.747183 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.747183 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:29.747183 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.747183 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.747183 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:29.747183 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.747183 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:29.747957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:29.748713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:29.748713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.748713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:29.748713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.748713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.748713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:29.748713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.748713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:29.749610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.750390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:29.751233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.751233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:29.805082 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:29.805911 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:29.806251 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:29.806983 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:29.807382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.808597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:29.808597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.808597 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:29.808597 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.808597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.808597 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:29.808597 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.808597 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.810245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:29.810721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:34.462486 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:34.464359 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:34.466341 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=259 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:25:34.495991 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:34.496423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.497577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:34.497577 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.497577 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:34.497577 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.497577 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.497577 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:34.497577 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.497577 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.499501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.500277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:34.620465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:34.622017 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:34.622415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:34.622732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.623654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:34.623654 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.623654 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:34.623654 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.623654 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.623654 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:34.623654 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.623654 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.625532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.626504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:34.649035 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:34.649035 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:34.649412 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:34.649708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.650382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:34.650382 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.650382 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:34.650382 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.650382 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.650382 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:34.650382 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.650382 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:34.651196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:34.651700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:34.651700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.651700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:34.651700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.651700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.651700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:34.651700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.651700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:34.652587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.653302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.654115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:34.654115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.654115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:34.744817 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:34.745465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:34.745775 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:34.746071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.746807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:34.746807 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.746807 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:34.746807 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.746807 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.746807 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:34.746807 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.746807 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:34.748039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:34.748718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:34.748718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.748718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:34.748718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.748718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.748718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:34.748718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.748718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:34.749794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.750255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:34.750687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.750687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:34.805757 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:34.806151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:34.806950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:34.807524 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:34.807837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.808816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:34.808816 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.808816 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:34.808816 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.808816 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.808816 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:34.808816 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.808816 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.810540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:34.811518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:39.463378 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:39.465119 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:39.497808 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:39.498158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.498656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:39.498656 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.498656 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:39.498656 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.498656 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.498656 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:39.498656 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.498656 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.499802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.500354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:39.621443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:39.622841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:39.623258 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:39.623635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.624780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:39.624780 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.624780 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:39.624780 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.624780 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.624780 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:39.624780 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.624780 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.626559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.627434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:39.649016 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:39.649748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:39.650145 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:39.650566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.651480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:39.651480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.651480 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:39.651480 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.651480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.651480 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:39.651480 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.651480 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:39.652841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:39.654036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:39.654036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.654036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:39.654036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.654036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.654036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:39.654036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.654036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:39.654611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.655147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.655694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:39.655694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.655694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:39.745608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:39.745977 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:39.746628 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:39.747199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.747988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:39.747988 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.747988 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:39.747988 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.747988 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.747988 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:39.747988 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.747988 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:39.748833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:39.749471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:39.749471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.749471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:39.749471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.749471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.749471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:39.749471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.749471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:39.750125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.750621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:39.751123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.751123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:39.806407 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:39.806800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:39.807334 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:39.807740 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:39.807962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.808759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:39.808759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.808759 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:39.808759 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.808759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.808759 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:39.808759 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.808759 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.810505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:39.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:44.463975 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:44.465278 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:44.498565 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:44.498818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.499410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:44.499410 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.499410 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:44.499410 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.499410 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.499410 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:44.499410 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.499410 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.500546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.501408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:44.622261 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:44.623075 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:44.623200 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:44.623330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.623772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:44.623772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.623772 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:44.623772 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.623772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.623772 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:44.623772 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.623772 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.624570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.625377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:44.649499 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:44.649873 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:44.656639 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:44.656987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.657958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:44.657958 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.657958 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:44.657958 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.657958 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.657958 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:44.657958 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.657958 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:44.659227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:44.659577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:44.659577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.659577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:44.659577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.659577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.659577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:44.659577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.659577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:44.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.660263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.660648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:44.660648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.660648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:44.746307 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:44.746763 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:44.747430 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:44.747834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.749142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:44.749142 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.749142 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:44.749142 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.749142 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.749142 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:44.749142 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.749142 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:44.750314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:44.751072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:44.751072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.751072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:44.751072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.751072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.751072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:44.751072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.751072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:44.752639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.753409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:44.754280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.754280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:44.806926 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:44.807489 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:44.807954 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:44.808165 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:44.808297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.809006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:44.809006 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.809006 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:44.809006 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.809006 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.809006 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:44.809006 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.809006 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.809802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:44.810575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:49.464868 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:49.469281 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:49.506946 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:49.507377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:49.510251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:49.510637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.511022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:49.511022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.511022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.511022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:49.511022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.511022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:49.622860 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:49.624160 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:49.624462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:49.624702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.625700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:49.625700 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.625700 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:49.625700 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.625700 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.625700 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:49.625700 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.625700 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.627611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.628332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:49.649961 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:49.650543 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:49.651119 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:49.651455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.652510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:49.652510 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.652510 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:49.652510 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.652510 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.652510 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:49.652510 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.652510 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:49.654035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:49.654567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:49.654567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.654567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:49.654567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.654567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.654567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:49.654567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.654567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:49.655187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.655955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.656477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:49.656477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.656477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:49.747527 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:49.747527 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:49.747527 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:49.747527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.748254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:49.748254 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.748254 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:49.748254 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.748254 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.748254 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:49.748254 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.748254 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:49.749665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.750692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:49.751335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:49.752042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:49.752042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:49.752042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.752042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:49.752042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.752042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.752042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:49.752042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.752042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:49.807678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:49.808249 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:49.808545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:49.808806 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:49.808958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.809546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:49.809546 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.809546 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:49.809546 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.809546 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.809546 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:49.809546 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.809546 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.810674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:49.811165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:54.466311 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:54.467494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:54.497354 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:54.497685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.498497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:54.498497 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.498497 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:54.498497 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.498497 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.498497 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:54.498497 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.498497 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.499856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.500671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:54.623490 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:54.624646 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:54.624880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:54.625032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.625942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:54.625942 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.625942 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:54.625942 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.625942 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.625942 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:54.625942 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.625942 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.627809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.628939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:54.650764 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:54.651253 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:54.651753 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:54.651968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.652554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:54.652554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.652554 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:54.652554 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.652554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.652554 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:54.652554 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.652554 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:54.653907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:54.654985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:54.654985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.654985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:54.654985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.654985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.654985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:54.654985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.654985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:54.655417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.656152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.657034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:54.657034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.657034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:54.747693 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:54.748328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:54.748708 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:54.749029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.749929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:54.749929 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.749929 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:54.749929 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.749929 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.749929 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:54.749929 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.749929 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:54.750857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:54.751354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:54.751354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.751354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:54.751354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.751354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.751354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:54.751354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.751354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:54.752326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.752800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:54.753317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.753317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:54.808250 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:54.808865 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:54.809435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:54.810015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:54.810310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.811860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:54.811860 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.811860 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:54.811860 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.811860 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.811860 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:54.811860 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.811860 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.814044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:54.815017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:59.467067 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:59.468546 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:59.502005 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:59.502406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.503353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:25:59.503353 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.503353 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:59.503353 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.503353 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.503353 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:59.503353 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.503353 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.504523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.505538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:59.624110 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:59.625481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:59.625636 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:59.625895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.626880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:59.626880 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.626880 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:59.626880 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.626880 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.626880 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:59.626880 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.626880 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.631244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.631634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:59.651206 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:59.651799 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:25:59.652386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:59.653355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.654180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:59.654180 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.654180 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:59.654180 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.654180 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.654180 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:59.654180 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.654180 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:59.655540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:59.655841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:59.655841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.655841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:59.655841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.655841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.655841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:59.655841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.655841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:25:59.657014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.657339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.657668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:59.657668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.657668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:59.748888 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:59.748888 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:59.749265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:59.749372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.749946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:59.749946 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.749946 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:59.749946 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.749946 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.749946 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:59.749946 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.749946 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:25:59.750981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.752063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:59.752495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:59.753050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:59.753050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:59.753050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.753050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:59.753050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.753050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.753050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:59.753050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.753050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:25:59.808977 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:25:59.809502 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:25:59.810005 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:59.810005 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:25:59.810270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.810827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:25:59.810827 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.810827 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:25:59.810827 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.810827 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.810827 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:25:59.810827 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.810827 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.813334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:25:59.814345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:04.467506 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:04.468680 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:04.523670 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:04.523947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.524677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:04.524677 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.524677 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:04.524677 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.524677 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.524677 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:04.524677 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.524677 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.525557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.526404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:04.624757 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:04.626333 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:04.626730 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:04.626997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.627902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:04.627902 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.627902 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:04.627902 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.627902 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.627902 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:04.627902 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.627902 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.629632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.630764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:04.651897 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:04.652365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:04.652862 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:04.653048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.653641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:04.653641 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.653641 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:04.653641 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.653641 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.653641 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:04.653641 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.653641 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:04.655260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:04.656623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:04.656623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.656623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:04.656623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.656623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.656623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:04.656623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.656623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:04.657595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.658598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.659524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:04.659524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.659524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:04.748801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:04.748971 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:04.749854 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:04.750414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.751474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:04.751474 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.751474 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:04.751474 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.751474 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.751474 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:04.751474 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.751474 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:04.752491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:04.753270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:04.753270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.753270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:04.753270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.753270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.753270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:04.753270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.753270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:04.753893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.754640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:04.755301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.755301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:04.809474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:04.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:04.810533 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:04.810819 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:04.810966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.811580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:04.811580 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.811580 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:04.811580 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.811580 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.811580 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:04.811580 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.811580 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.812572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:04.813251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:09.468115 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:09.469818 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:09.506699 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:09.506970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.507754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:09.507754 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.507754 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:09.507754 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.507754 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.507754 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:09.507754 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.507754 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.508968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.509826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:09.625541 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:09.626531 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:09.626799 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:09.627009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.627855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:09.627855 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.627855 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:09.627855 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.627855 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.627855 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:09.627855 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.627855 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.629417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.630208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:09.652408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:09.652969 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:09.653601 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:09.653883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.654682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:09.654682 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.654682 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:09.654682 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.654682 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.654682 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:09.654682 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.654682 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:09.656099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:09.656974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:09.656974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.656974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:09.656974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.656974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.656974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:09.656974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.656974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:09.658815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.661711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.662654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:09.662654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.662654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:09.749464 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:09.749981 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:09.750627 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:09.750903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.751679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:09.751679 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.751679 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:09.751679 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.751679 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.751679 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:09.751679 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.751679 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:09.752548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:09.753203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:09.753203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.753203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:09.753203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.753203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.753203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:09.753203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.753203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:09.753555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.754432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:09.755181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.755181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:09.809990 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:09.810503 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:09.810975 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:09.811814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:09.812231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.812976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:09.812976 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.812976 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:09.812976 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.812976 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.812976 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:09.812976 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.812976 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:09.815685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:14.469009 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:14.470162 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:14.506198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.515531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:14.516554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:14.517587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.517587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:14.517587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.517587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.517587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:14.517587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.517587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:14.626154 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:14.627436 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:14.627760 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:14.627993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.628732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:14.628732 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.628732 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:14.628732 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.628732 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.628732 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:14.628732 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.628732 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.629950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.630559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:14.652894 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:14.653327 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:14.653816 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:14.654085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.654873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:14.654873 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.654873 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:14.654873 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.654873 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.654873 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:14.654873 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.654873 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:14.656000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:14.656888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:14.656888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.656888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:14.656888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.656888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.656888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:14.656888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.656888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:14.657717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.658298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.658828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:14.658828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.658828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:14.751082 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:14.751082 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:14.751515 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:14.751919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:14.754644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:14.755889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.756288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:14.756666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.756666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:14.810832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:14.811499 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:14.812021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:14.812426 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:14.812672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.813419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:14.813419 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.813419 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:14.813419 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.813419 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.813419 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:14.813419 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.813419 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.815330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:14.815833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:19.469820 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:19.471834 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:19.510256 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:19.510751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.511686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:19.511686 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.511686 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:19.511686 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.511686 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.511686 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:19.511686 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.511686 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.512835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.513408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:19.626767 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:19.628534 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:19.628963 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:19.629313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.630662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:19.630662 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.630662 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:19.630662 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.630662 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.630662 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:19.630662 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.630662 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.632037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.633080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:19.653443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:19.653831 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:19.654346 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:19.654723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.655448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:19.655448 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.655448 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:19.655448 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.655448 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.655448 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:19.655448 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.655448 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:19.656793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:19.657491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:19.657491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.657491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:19.657491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.657491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.657491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:19.657491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.657491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:19.658457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.659195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:19.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.659907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:19.757613 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:19.758522 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:19.759550 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:19.760133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.762387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:19.762387 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.762387 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:19.762387 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.762387 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.762387 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:19.762387 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.762387 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:19.764113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:19.764458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:19.764458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.764458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:19.764458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.764458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.764458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:19.764458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.764458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:19.765873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.766254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:19.766624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.766624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:19.837416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:19.838053 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:19.838808 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:19.839331 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:19.839453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.840779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:19.840779 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.840779 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:19.840779 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.840779 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.840779 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:19.840779 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.840779 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.843036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:19.843729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:24.470582 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:24.472052 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:24.502557 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:24.502981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.503999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:24.503999 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.503999 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:24.503999 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.503999 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.503999 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:24.503999 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.503999 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.504942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.505459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:24.627539 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:24.629122 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:24.629586 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:24.629865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.630933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:24.630933 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.630933 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:24.630933 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.630933 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.630933 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:24.630933 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.630933 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.632172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.632987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:24.653902 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:24.654348 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:24.654684 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:24.654867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.655346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:24.655346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.655346 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:24.655346 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.655346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.655346 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:24.655346 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.655346 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:24.655897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.656667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:24.657397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:24.658170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:24.658170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:24.658170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:24.658170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:24.658170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.658170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:24.658170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.658170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.658170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:24.658170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.658170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:24.759021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:24.759021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:24.759774 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:24.760210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.761146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:24.761146 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.761146 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:24.761146 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.761146 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.761146 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:24.761146 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.761146 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:24.761951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.763027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:24.763889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:24.764800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:24.764800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:24.764800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.764800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:24.764800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.764800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.764800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:24.764800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.764800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:24.838134 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:24.838705 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:24.839386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:24.839800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:24.840022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.840895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:24.840895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.840895 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:24.840895 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.840895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.840895 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:24.840895 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.840895 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.842635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:24.843139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:29.471447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:29.472747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:29.511660 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:29.511660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.512406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:29.512406 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.512406 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:29.512406 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.512406 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.512406 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:29.512406 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.512406 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.513489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.514057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:29.628479 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:29.629910 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:29.630268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:29.630498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.631381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:29.631381 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.631381 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:29.631381 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.631381 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.631381 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:29.631381 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.631381 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.632513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.633380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:29.654558 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:29.654905 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:29.655308 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:29.655578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.656060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:29.656060 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.656060 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:29.656060 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.656060 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.656060 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:29.656060 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.656060 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:29.656842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.658043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:29.659140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:29.660459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:29.660459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:29.660459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:29.660459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:29.660459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.660459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:29.660459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.660459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.660459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:29.660459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.660459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:29.758962 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:29.759347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:29.759837 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:29.760202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.761180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:29.761180 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.761180 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:29.761180 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.761180 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.761180 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:29.761180 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.761180 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:29.762591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:29.763323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:29.763323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.763323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:29.763323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.763323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.763323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:29.763323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.763323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:29.764120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.765258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:29.766064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.766064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:29.839095 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:29.839095 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:29.839505 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:29.839849 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:29.840071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.842511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:29.843644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:29.844536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:29.844536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:29.844536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:29.844536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:34.472184 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:34.474151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:34.476277 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:26:34.516413 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:34.516728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.517448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:34.517448 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.517448 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:34.517448 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.517448 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.517448 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:34.517448 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.517448 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.518643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.519010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:34.629046 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:34.630125 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:34.630488 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:34.630758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.631635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:34.631635 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.631635 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:34.631635 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.631635 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.631635 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:34.631635 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.631635 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.632805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.633299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:34.654955 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:34.655386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:34.656586 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:34.656586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.657468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:34.657468 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.657468 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:34.657468 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.657468 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.657468 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:34.657468 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.657468 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:34.658918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:34.659498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:34.659498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.659498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:34.659498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.659498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.659498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:34.659498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.659498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:34.660432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.660905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.661455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:34.661455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.661455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:34.759556 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:34.759872 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:34.760747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:34.761159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.761997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:34.761997 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.761997 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:34.761997 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.761997 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.761997 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:34.761997 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.761997 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:34.763424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:34.764159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:34.764159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.764159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:34.764159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.764159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.764159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:34.764159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.764159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:34.764978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.765868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:34.766759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.766759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:34.839614 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:34.840183 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:34.840651 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:34.840934 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:34.841099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.841807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:34.841807 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.841807 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:34.841807 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.841807 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.841807 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:34.841807 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.841807 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.843129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:34.844770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:39.472966 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:39.474131 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:39.515015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:39.515321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.515996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:39.515996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.515996 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:39.515996 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.515996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.515996 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:39.515996 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.515996 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.517178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.518043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:39.629790 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:39.630960 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:39.631342 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:39.631630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.632465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:39.632465 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.632465 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:39.632465 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.632465 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.632465 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:39.632465 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.632465 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.633641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.634576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:39.655352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:39.655624 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:39.656093 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:39.656428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.657200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:39.657200 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.657200 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:39.657200 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.657200 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.657200 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:39.657200 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.657200 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:39.658907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:39.659693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:39.659693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.659693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:39.659693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.659693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.659693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:39.659693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.659693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:39.660622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.661437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.662375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:39.662375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.662375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:39.760236 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:39.760941 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:39.761494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:39.761870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.762619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:39.762619 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.762619 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:39.762619 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.762619 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.762619 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:39.762619 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.762619 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:39.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:39.764366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:39.764366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.764366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:39.764366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.764366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.764366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:39.764366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.764366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:39.764793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.765580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:39.766531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.766531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:39.840249 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:39.840928 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:39.841504 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:39.841859 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:39.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.842824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:39.842824 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.842824 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:39.842824 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.842824 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.842824 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:39.842824 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.842824 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.844669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:39.845450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:44.473686 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:44.475583 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:44.512059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:44.512390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.513317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:44.513317 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.513317 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:44.513317 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.513317 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.513317 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:44.513317 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.513317 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.514245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.515118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:44.630461 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:44.631859 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:44.632019 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:44.632872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.633070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:44.633070 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.633070 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:44.633070 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.633070 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.633070 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:44.633070 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.633070 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.635059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.635915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:44.655854 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:44.656261 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:44.657192 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:44.657192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.657964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:44.657964 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.657964 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:44.657964 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.657964 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.657964 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:44.657964 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.657964 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:44.658805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:44.659206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:44.659206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.659206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:44.659206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.659206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.659206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:44.659206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.659206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:44.659797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.660148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.660666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:44.660666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.660666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:44.761085 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:44.761606 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:44.762694 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:44.762694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.763167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:44.763167 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.763167 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:44.763167 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.763167 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.763167 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:44.763167 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.763167 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:44.764530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:44.764995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:44.764995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.764995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:44.764995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.764995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.764995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:44.764995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.764995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:44.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.766645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:44.767237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.767237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:44.840835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:44.841091 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:44.841676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:44.841900 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:44.841996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.842485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:44.842485 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.842485 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:44.842485 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.842485 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.842485 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:44.842485 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.842485 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.843245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:44.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:49.475185 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:49.476045 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:49.518005 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:49.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.518918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:49.518918 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.518918 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:49.518918 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.518918 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.518918 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:49.518918 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.518918 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.520147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.521021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:49.631958 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:49.632330 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:49.632539 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:49.632681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.634515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:49.634515 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.634515 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:49.634515 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.634515 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.634515 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:49.634515 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.634515 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.636950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.637866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:49.656453 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:49.659012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:49.659568 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:49.659883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.660608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:49.660608 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.660608 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:49.660608 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.660608 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.660608 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:49.660608 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.660608 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:49.661302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.662177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:49.662971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:49.663880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:49.663880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:49.663880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:49.663880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:49.663880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.663880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:49.663880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.663880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.663880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:49.663880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.663880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:49.761840 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:49.763457 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:49.763974 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:49.764278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.765783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:49.765783 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.765783 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:49.765783 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.765783 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.765783 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:49.765783 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.765783 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:49.767219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:49.767539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:49.767539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.767539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:49.767539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.767539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.767539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:49.767539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.767539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:49.769089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.769458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:49.769819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.769819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:49.841410 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:49.841953 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:49.842678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:49.842918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:49.843069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.843751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:49.843751 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.843751 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:49.843751 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.843751 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.843751 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:49.843751 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.843751 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:49.845810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:54.475161 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:54.477747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:54.513362 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:54.513829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.514941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:54.514941 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.514941 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:54.514941 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.514941 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.514941 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:54.514941 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.514941 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.516228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.516775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:54.631698 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:54.632885 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:54.633147 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:54.633296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.634062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:54.634062 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.634062 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:54.634062 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.634062 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.634062 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:54.634062 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.634062 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.635617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.636452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:54.658666 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:54.658887 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:54.659775 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:54.660155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.661395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:54.661395 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.661395 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:54.661395 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.661395 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.661395 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:54.661395 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.661395 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:54.663050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:54.663577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:54.663577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.663577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:54.663577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.663577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.663577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:54.663577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.663577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:54.664946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.665405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.665849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:54.665849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.665849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:54.763461 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:54.763892 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:54.764691 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:54.765106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.765983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:54.765983 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.765983 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:54.765983 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.765983 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.765983 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:54.765983 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.765983 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:54.767430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:54.767898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:54.767898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.767898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:54.767898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.767898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.767898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:54.767898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.767898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:54.769128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.769602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:54.770068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.770068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:54.841940 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:54.842427 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:54.842804 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:54.843140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:54.843403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.844391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:54.844391 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.844391 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:54.844391 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.844391 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.844391 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:54.844391 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.844391 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.846776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:54.847519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:59.476118 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:59.478513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:59.515801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:59.516336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.517609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:26:59.517609 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.517609 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:59.517609 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.517609 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.517609 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:59.517609 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.517609 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.518949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.519931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:59.632257 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:59.633563 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:59.633847 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:59.634065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.634953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:59.634953 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.634953 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:59.634953 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.634953 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.634953 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:59.634953 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.634953 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.636514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.637375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:59.659100 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:59.659609 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:26:59.660168 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:59.660499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.661232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:59.661232 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.661232 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:59.661232 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.661232 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.661232 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:59.661232 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.661232 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:59.662450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:59.663177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:59.663177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.663177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:59.663177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.663177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.663177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:59.663177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.663177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:26:59.663948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.664640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.665487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:59.665487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.665487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:59.764295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:59.764804 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:59.765676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:59.766117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.767043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:59.767043 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.767043 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:59.767043 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.767043 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.767043 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:59.767043 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.767043 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:26:59.768492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:59.768948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:26:59.768948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.768948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:26:59.768948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.768948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.768948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:26:59.768948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.768948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:26:59.770267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.771035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:59.771938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.771938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:26:59.842656 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:26:59.843242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:26:59.843877 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:59.844225 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:26:59.844502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.845350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:26:59.845350 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.845350 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:26:59.845350 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.845350 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.845350 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:26:59.845350 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.845350 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:26:59.848804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:04.477106 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:04.478829 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:04.518798 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:04.519335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.520424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:04.520424 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.520424 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:04.520424 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.520424 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.520424 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:04.520424 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.520424 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.521886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.523549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:04.633039 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:04.634468 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:04.634793 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:04.635035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.635916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:04.635916 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.635916 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:04.635916 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.635916 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.635916 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:04.635916 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.635916 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.637700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.638502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:04.659653 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:04.660357 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:04.660742 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:04.661018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.662198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:04.662198 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.662198 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:04.662198 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.662198 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.662198 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:04.662198 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.662198 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:04.663118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:04.663656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:04.663656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.663656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:04.663656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.663656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.663656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:04.663656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.663656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:04.663972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.664452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.664954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:04.664954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.664954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:04.765007 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:04.765650 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:04.766165 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:04.766439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.766937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:04.766937 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.766937 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:04.766937 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.766937 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.766937 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:04.766937 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.766937 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:04.767913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:04.768654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:04.768654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.768654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:04.768654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.768654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.768654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:04.768654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.768654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:04.769348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.770196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:04.771126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.771126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:04.843403 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:04.843832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:04.844300 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:04.845196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:04.845196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.845361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:04.845361 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.845361 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:04.845361 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.845361 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.845361 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:04.845361 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.845361 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.846975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:04.847881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:09.477808 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:09.480716 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:09.522774 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:09.523082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.524075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:09.524075 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.524075 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:09.524075 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.524075 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.524075 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:09.524075 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.524075 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.524861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.525354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:09.633735 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:09.634942 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:09.635327 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:09.635578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.636552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:09.636552 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.636552 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:09.636552 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.636552 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.636552 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:09.636552 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.636552 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.640104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.640997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:09.660248 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:09.660708 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:09.661521 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:09.661682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.662387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:09.662387 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.662387 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:09.662387 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.662387 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.662387 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:09.662387 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.662387 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:09.663542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:09.663958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:09.663958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.663958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:09.663958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.663958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.663958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:09.663958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.663958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:09.664559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.665422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.665896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:09.665896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.665896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:09.765834 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:09.766421 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:09.766967 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:09.767380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.768121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:09.768121 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.768121 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:09.768121 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.768121 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.768121 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:09.768121 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.768121 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:09.769506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:09.769856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:09.769856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.769856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:09.769856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.769856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.769856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:09.769856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.769856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:09.771144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.771931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:09.776667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.776667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:09.844032 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:09.844664 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:09.845135 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:09.845323 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:09.845473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.846391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:09.846391 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.846391 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:09.846391 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.846391 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.846391 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:09.846391 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.846391 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.847981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:09.849982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:14.480372 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:14.481611 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:14.513646 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:14.513868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.514433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:14.514433 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.514433 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:14.514433 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.514433 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.514433 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:14.514433 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.514433 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.515304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.515794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:14.634578 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:14.636033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:14.636236 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:14.636375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.641267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:14.641661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.642035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.642035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:14.642035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.642035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:14.660580 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:14.661108 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:14.662358 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:14.662635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.663415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:14.663415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.663415 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:14.663415 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.663415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.663415 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:14.663415 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.663415 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:14.664811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.667817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:14.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:14.668534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:14.668534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:14.668534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:14.668534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:14.668534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.668534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:14.668534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.668534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.668534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:14.668534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.668534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:14.766502 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:14.767240 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:14.767664 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:14.767944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:14.770806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.771352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:14.771724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.772193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:14.845255 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:14.845828 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:14.846362 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:14.846641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:14.846795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.847559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:14.847559 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.847559 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:14.847559 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.847559 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.847559 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:14.847559 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.847559 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.849633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:14.850843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:19.481122 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:19.482373 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:19.515638 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:19.515980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.516712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:19.516712 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.516712 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:19.516712 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.516712 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.516712 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:19.516712 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.516712 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.517630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.518176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:19.635209 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:19.636438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:19.636717 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:19.636955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.637953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:19.637953 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.637953 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:19.637953 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.637953 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.637953 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:19.637953 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.637953 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.640046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.640951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:19.661140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:19.661523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:19.662101 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:19.662436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.663130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:19.663130 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.663130 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:19.663130 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.663130 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.663130 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:19.663130 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.663130 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:19.664102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.665094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:19.665931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:19.666794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:19.666794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:19.666794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:19.666794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:19.666794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.666794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:19.666794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.666794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.666794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:19.666794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.666794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:19.767424 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:19.767817 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:19.768508 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:19.768948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.769801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:19.769801 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.769801 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:19.769801 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.769801 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.769801 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:19.769801 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.769801 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:19.771184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:19.772001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:19.772001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.772001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:19.772001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.772001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.772001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:19.772001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.772001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:19.772965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.773439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:19.773904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.773904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:19.846298 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:19.846513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:19.847259 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:19.847782 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:19.848041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.848961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:19.848961 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.848961 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:19.848961 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.848961 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.848961 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:19.848961 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.848961 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.850627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:19.851326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:24.481796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:24.483027 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:24.514789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:24.515116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.516015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:24.516015 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.516015 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:24.516015 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.516015 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.516015 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:24.516015 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.516015 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.516936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.517731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:24.635857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:24.637352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:24.637690 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:24.637934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.639379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:24.639379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.639379 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:24.639379 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.639379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.639379 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:24.639379 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.639379 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.640786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.641553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:24.661535 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:24.662076 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:24.662737 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:24.663034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.663964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:24.663964 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.663964 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:24.663964 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.663964 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.663964 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:24.663964 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.663964 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:24.665404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:24.666102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:24.666102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.666102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:24.666102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.666102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.666102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:24.666102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.666102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:24.666963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.667635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.668511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:24.668511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.668511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:24.768023 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:24.768407 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:24.768957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:24.769365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.770035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:24.770035 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.770035 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:24.770035 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.770035 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.770035 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:24.770035 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.770035 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:24.771237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:24.771972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:24.771972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.771972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:24.771972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.771972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.771972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:24.771972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.771972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:24.773052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.773816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:24.775084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.775084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:24.846599 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:24.846980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:24.847496 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:24.847846 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:24.848063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.848821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:24.848821 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.848821 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:24.848821 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.848821 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.848821 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:24.848821 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.848821 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.850612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:24.851401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:29.484743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:29.485804 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:29.518906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:29.519174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.520019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:29.520019 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.520019 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:29.520019 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.520019 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.520019 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:29.520019 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.520019 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.520941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.521506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:29.636702 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:29.638013 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:29.638205 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:29.638344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.639039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:29.639039 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.639039 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:29.639039 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.639039 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.639039 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:29.639039 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.639039 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.640916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.641806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:29.662037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:29.662563 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:29.663097 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:29.663427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.664136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:29.664136 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.664136 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:29.664136 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.664136 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.664136 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:29.664136 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.664136 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:29.665409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:29.666189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:29.666189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.666189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:29.666189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.666189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.666189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:29.666189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.666189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:29.666933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.667676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.668413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:29.668413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.668413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:29.768770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:29.769052 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:29.769641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:29.769999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.770829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:29.770829 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.770829 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:29.770829 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.770829 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.770829 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:29.770829 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.770829 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:29.772075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:29.772840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:29.772840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.772840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:29.772840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.772840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.772840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:29.772840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.772840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:29.773113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.773704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:29.774260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.774260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:29.847158 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:29.847674 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:29.848154 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:29.848540 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:29.848766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.849473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:29.849473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.849473 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:29.849473 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.849473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.849473 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:29.849473 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.849473 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.850660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:29.851518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:34.485591 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:34.487161 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:34.489013 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:27:34.520177 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:34.521015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.521487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:34.521487 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.521487 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:34.521487 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.521487 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.521487 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:34.521487 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.521487 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.522429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.522799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:34.637494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:34.638667 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:34.638914 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:34.639527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.639813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:34.639813 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.639813 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:34.639813 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.639813 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.639813 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:34.639813 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.639813 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.641992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.643073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:34.662616 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:34.662999 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:34.663556 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:34.663878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.664665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:34.664665 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.664665 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:34.664665 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.664665 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.664665 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:34.664665 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.664665 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:34.666009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.667832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:34.668615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:34.669762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:34.669762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:34.669762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:34.669762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:34.669762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.669762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:34.669762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.669762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.669762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:34.669762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.669762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:34.769339 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:34.769636 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:34.770244 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:34.770704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.771484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:34.771484 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.771484 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:34.771484 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.771484 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.771484 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:34.771484 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.771484 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:34.772128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.772845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:34.773587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:34.774432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:34.774432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:34.774432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.774432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:34.774432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.774432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.774432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:34.774432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.774432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:34.847846 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:34.848613 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:34.849471 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:34.849944 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:34.850197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.851104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:34.851104 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.851104 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:34.851104 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.851104 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.851104 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:34.851104 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.851104 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.852762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:34.853606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:39.356046 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:39.364064 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:389}}
Mar 23 22:27:39.364153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, security_groups=2, routers=1, router_ports=1 {{(pid=104232) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:370}}
Mar 23 22:27:39.364283 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Fixing resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:27:39.398507 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, resource={'id': '06cffee1-b8de-433e-a67a-ec0e950c9238', 'name': 'private', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['130adef4-306b-4397-a096-91ee5094d0e2', 'fb1912da-e258-4d41-8cd4-7896f0de6385'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:17:25Z', 'updated_at': '2026-03-23T22:17:28Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 36968}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.398868 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.399499 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c4286b-d53b-4217-a9f3-05177afa9297, external_ids_update=None, columns={'tag': 36968}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.399764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.400676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:39.400676 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.400676 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:39.400676 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.400676 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.400676 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:39.400676 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.400676 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:39.401863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Failed to fix resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.401863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:27:39.401863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:27:39.401863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:27:39.401863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:27:39.401863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:27:39.401863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:27:39.401863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:39.401863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.401863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:39.401863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.402126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.402407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:27:39.402407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.402407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:27:39.402407 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Fixing resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:27:39.428550 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, resource={'id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'name': 'public', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'a53179c9-417a-4560-9bc7-e3f64551b15b'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-23T22:17:31Z', 'updated_at': '2026-03-23T22:17:40Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.428959 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.429652 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a8bd993e-d23e-4e73-8066-766307763bdb, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.429998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.430844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:39.430844 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.430844 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:39.430844 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.430844 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.430844 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:39.430844 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.430844 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:39.432114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Failed to fix resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.432114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:27:39.432114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:27:39.432114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:27:39.432114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:27:39.432114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:27:39.432114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:27:39.432114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:39.432114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.432114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:39.432114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.432678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.433245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:27:39.433245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.433245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:27:39.433245 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Fixing resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:27:39.459365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, resource={'id': '3a042d54-7a79-465e-bd57-47cd6833a4d0', 'name': 'admin_net', 'tenant_id': 'b4ffb2af173e4612a30978ef510f4f44', 'project_id': 'b4ffb2af173e4612a30978ef510f4f44', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['935d20c1-7d6b-489a-8358-b5fb94945b7e'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:20:57Z', 'updated_at': '2026-03-23T22:20:58Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 47875}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.459755 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.460533 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c442d0-efff-4c7e-bcb6-377b4da39c34, external_ids_update=None, columns={'tag': 47875}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.460533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.461246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:39.461246 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.461246 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:39.461246 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.461246 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.461246 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:39.461246 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.461246 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:39.462044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Failed to fix resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.462044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:27:39.462044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:27:39.462044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:27:39.462044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:27:39.462044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:27:39.462044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:27:39.462044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:39.462044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.462044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:39.462044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.462387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.462714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:27:39.462714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.462714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:27:39.462714 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Fixing resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:27:39.486255 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:39.488762 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, resource={'id': 'c9e6c30a-b30e-473b-b06a-50be75c90b36', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['421749b7-bb0d-4854-b3f4-f0414f4c37fb'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:21:34Z', 'updated_at': '2026-03-23T22:21:35Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 48210}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.489513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:39.490240 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.490604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13535482-a705-4453-af87-e8a7d8c10c82, external_ids_update=None, columns={'tag': 48210}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.490739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.491376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:39.491376 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.491376 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:39.491376 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.491376 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.491376 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:39.491376 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.491376 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:39.492708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Failed to fix resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.492708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:27:39.492708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:27:39.492708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:27:39.492708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:27:39.492708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:27:39.492708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:27:39.492708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:39.492708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.492708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:39.492708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.493027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.493375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:27:39.493375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.493375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:27:39.493375 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Fixing resource f247eeb9-d9b1-471b-a832-4f861c1fe631 (type: security_groups) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:27:39.557327 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.557837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.558855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:27:39.558855 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.558855 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:39.558855 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.558855 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.558855 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:39.558855 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.558855 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.560136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.560971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:39.638370 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:39.639979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.640307 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.640474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.642701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:39.643258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.643785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.643785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:39.643785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.643785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:39.663120 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:39.663587 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:39.664881 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.665394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.666411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:39.666411 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.666411 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:39.666411 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.666411 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.666411 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:39.666411 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.666411 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:39.667455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.669536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:39.670416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:39.672521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:39.672521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:39.672521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:39.672521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:39.672521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.672521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:39.672521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.672521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.672521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:39.672521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.672521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:39.769845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:39.770419 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:39.771099 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.771621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.772477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:39.772477 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.772477 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:39.772477 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.772477 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.772477 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:39.772477 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.772477 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:39.773244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.774344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:39.775053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:39.775949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:39.775949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:39.775949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.775949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:39.775949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.775949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.775949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:39.775949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.775949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:39.848696 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:39.849401 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:39.850260 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.850777 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:39.851030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.852045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:39.852045 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.852045 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:39.852045 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.852045 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.852045 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:39.852045 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.852045 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.877339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:39.878310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:39.938011 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.db.ovn_revision_numbers_db [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Successfully bumped revision number for resource f247eeb9-d9b1-471b-a832-4f861c1fe631 (type: security_groups) to 10
Mar 23 22:27:39.938011 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Fixing resource 6bedf0ff-4721-48bb-95d0-b022041d2807 (type: security_groups) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:27:39.982574 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.db.ovn_revision_numbers_db [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Successfully bumped revision number for resource 6bedf0ff-4721-48bb-95d0-b022041d2807 (type: security_groups) to 15
Mar 23 22:27:39.982719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Fixing resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:27:40.059508 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, resource={'id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'enable_snat': True}, 'gw_port_id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-t62km', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'ip_address': '172.24.5.146', 'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a'}, {'ip_address': '2001:db8::396', 'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 3, 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d'}, resource_type=routers, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:40.059920 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:40.060268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:40.060399 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:40.060596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:40.061579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:40.061579 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:40.061579 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:40.061579 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:40.061579 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:40.061579 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:40.061579 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:40.061579 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:40.061943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Unable to update router 154c08d6-78c5-49fd-a785-0df3f4277a23. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:40.063339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Failed to fix resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:40.063339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:27:40.063339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:27:40.063339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:27:40.063339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:27:40.063339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:27:40.063339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 23 22:27:40.063339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 23 22:27:40.063339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 23 22:27:40.063339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 23 22:27:40.063339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:27:40.064351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:27:40.064967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:40.064967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:27:40.064967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:40.064967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:27:40.064967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:27:40.064967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:40.064967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:27:40.064967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:40.064967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:27:40.064967 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Fixing resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:27:40.193599 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, resource={'id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'name': '', 'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'project_id': '', 'mac_address': 'fa:16:3e:aa:0e:57', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160382', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:40.193835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a a53179c9-417a-4560-9bc7-e3f64551b15b', 'neutron:network_name': 'neutron-bb9d5c47-4694-42c2-a018-18e3c9058915', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-154c08d6-78c5-49fd-a785-0df3f4277a23'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::396/64', '172.24.5.146/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:40.194407 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=348aba7c-88a8-4a19-9486-df3f7edb4007, lrouter_port=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:40.194521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:40.195471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Traceback (most recent call last):
Mar 23 22:27:40.195471 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:40.195471 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:40.195471 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:40.195471 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:40.195471 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:40.195471 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:40.195471 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:40.196633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Failed to fix resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:40.196633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:27:40.196633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:27:40.196633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:27:40.196633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:27:40.196633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:27:40.196633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 23 22:27:40.196633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:40.196633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:40.196633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:40.196633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:27:40.197247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:40.197806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:27:40.197806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:40.197806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:27:40.197806 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f206b71-0be9-40b4-9da9-c23c9fe5332d None None] Maintenance task: Synchronization completed (took 0.83 seconds)
Mar 23 22:27:44.487615 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:44.488545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:44.518122 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:44.518356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.518883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:27:44.518883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.518883 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:44.518883 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.518883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.518883 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:44.518883 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.518883 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.519793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.520596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:44.639246 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:44.640296 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:44.640504 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:44.640623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.641234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:44.641234 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.641234 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:44.641234 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.641234 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.641234 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:44.641234 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.641234 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.642405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.642903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:44.663566 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:44.664146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:44.664619 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:44.664812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.665280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:44.665280 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.665280 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:44.665280 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.665280 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.665280 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:44.665280 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.665280 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:44.666372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.667054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:44.667597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:44.668111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:44.668111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:44.668111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:44.668111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:44.668111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.668111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:44.668111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.668111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.668111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:44.668111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.668111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:44.770408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:44.770747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:44.771419 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:44.771710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.772394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:44.772394 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.772394 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:44.772394 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.772394 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.772394 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:44.772394 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.772394 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:44.773616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:44.774149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:44.774149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.774149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:44.774149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.774149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.774149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:44.774149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.774149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:44.776369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.776993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:44.777554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.777554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:44.849340 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:44.850135 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:44.850837 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:44.851335 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:44.851683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.852439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:44.852439 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.852439 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:44.852439 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.852439 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.852439 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:44.852439 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.852439 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.854082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:44.854817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:49.488373 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:49.489953 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:49.525564 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.556993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:49.558056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:49.559198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.559198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:49.559198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.559198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.559198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:49.559198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.559198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:49.639855 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:49.641432 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:49.641812 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:49.642086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.643014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:49.643014 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.643014 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:49.643014 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.643014 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.643014 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:49.643014 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.643014 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.645158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.645957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:49.664180 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:49.664607 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:49.665808 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:49.665808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.666099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:49.666099 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.666099 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:49.666099 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.666099 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.666099 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:49.666099 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.666099 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:49.666563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.667825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:49.668339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:49.668967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:49.668967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:49.668967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:49.668967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:49.668967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.668967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:49.668967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.668967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.668967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:49.668967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.668967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:49.771245 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:49.771657 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:49.772468 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:49.773051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.774309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:49.774309 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.774309 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:49.774309 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.774309 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.774309 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:49.774309 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.774309 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:49.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:49.774921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:49.774921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.774921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:49.774921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.774921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.774921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:49.774921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.774921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:49.775619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.775975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:49.776329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.776329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:49.850006 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:49.850356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:49.850974 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:49.851249 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:49.851424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.852063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:49.852063 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.852063 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:49.852063 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.852063 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.852063 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:49.852063 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.852063 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.853483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:49.854021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:54.496790 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:54.496790 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:54.523802 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:54.524035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.524617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:27:54.524617 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.524617 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:54.524617 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.524617 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.524617 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:54.524617 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.524617 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.525516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.526267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:54.640583 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:54.641702 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:54.642197 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:54.642497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.643586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:54.643586 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.643586 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:54.643586 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.643586 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.643586 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:54.643586 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.643586 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.644840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.645559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:54.664711 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:54.665245 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:54.665944 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:54.666664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.667365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:54.667365 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.667365 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:54.667365 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.667365 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.667365 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:54.667365 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.667365 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:54.668636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:54.670062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:54.670062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.670062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:54.670062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.670062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.670062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:54.670062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.670062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:54.670450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.671167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.671971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:54.671971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.671971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:54.771710 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:54.772275 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:54.772909 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:54.773394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.774259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:54.774259 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.774259 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:54.774259 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.774259 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.774259 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:54.774259 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.774259 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:54.775683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:54.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:54.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:54.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:54.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:54.777584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.778289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:54.778972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.778972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:54.850672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:54.851337 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:54.852068 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:54.852524 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:54.852760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.853809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:54.853809 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.853809 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:54.853809 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.853809 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.853809 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:54.853809 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.853809 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.855952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:54.856838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:59.490406 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:59.491796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:59.531032 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:59.532907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.533991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:27:59.533991 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.533991 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:59.533991 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.533991 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.533991 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:59.533991 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.533991 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.535289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.536275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:59.641550 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:59.641945 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:59.642133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:59.642318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.642773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:59.642773 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.642773 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:59.642773 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.642773 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.642773 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:59.642773 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.642773 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.643845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.644452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:59.665166 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:59.665762 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:27:59.666328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:59.666641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.667387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:59.667387 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.667387 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:59.667387 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.667387 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.667387 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:59.667387 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.667387 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:59.668602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:59.669162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:59.669162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.669162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:59.669162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.669162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.669162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:59.669162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.669162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:27:59.670161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.670696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.671338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:59.671338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.671338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:59.772162 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:59.772985 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:59.773657 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:59.774030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.774741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:59.774741 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.774741 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:59.774741 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.774741 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.774741 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:59.774741 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.774741 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:27:59.775974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:59.776790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:27:59.776790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.776790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:27:59.776790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.776790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.776790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:27:59.776790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.776790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:27:59.777542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.779387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:59.780490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.780490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:27:59.851143 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:27:59.851582 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:27:59.851916 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:59.852145 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:27:59.852362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.852874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:27:59.852874 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.852874 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:27:59.852874 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.852874 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.852874 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:27:59.852874 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.852874 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.853714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:27:59.854858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:04.491514 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:04.492265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:04.521120 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:04.521120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.521602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:04.521602 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.521602 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:04.521602 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.521602 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.521602 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:04.521602 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.521602 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.522481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.522847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:04.641718 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:04.642790 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:04.642790 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:04.642992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.643473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:04.643473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.643473 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:04.643473 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.643473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.643473 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:04.643473 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.643473 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.644483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.645351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:04.666818 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:04.667292 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:04.667910 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:04.668377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.669267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:04.669267 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.669267 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:04.669267 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.669267 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.669267 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:04.669267 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.669267 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:04.670806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:04.671555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:04.671555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.671555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:04.671555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.671555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.671555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:04.671555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.671555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:04.672001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.672654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.673509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:04.673509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.673509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:04.773020 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:04.773422 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:04.774346 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:04.774754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.775508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:04.775508 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.775508 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:04.775508 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.775508 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.775508 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:04.775508 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.775508 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:04.776161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:04.776744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:04.776744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.776744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:04.776744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.776744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.776744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:04.776744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.776744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:04.776994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.777486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:04.778053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.778053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:04.851674 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:04.852558 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:04.853248 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:04.853670 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:04.853927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.856083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:04.856755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:04.857283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:04.857283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:04.857283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:04.857283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:09.492127 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:09.494676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:09.538340 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:09.538555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.539168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:09.539168 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.539168 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:09.539168 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.539168 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.539168 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:09.539168 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.539168 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.540506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.541079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:09.642195 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:09.644242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:09.644242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:09.644242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.644242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:09.644242 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.644242 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:09.644242 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.644242 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.644242 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:09.644242 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.644242 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.644827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.645166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:09.667244 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:09.669598 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:09.669906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:09.670069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.670467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:09.670467 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.670467 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:09.670467 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.670467 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.670467 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:09.670467 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.670467 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:09.670967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:09.671482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:09.671482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.671482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:09.671482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.671482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.671482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:09.671482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.671482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:09.671708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.672202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.672755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:09.672755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.672755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:09.773620 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:09.774655 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:09.776176 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:09.776638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.777321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:09.777321 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.777321 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:09.777321 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.777321 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.777321 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:09.777321 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.777321 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:09.777858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:09.778237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:09.778237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.778237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:09.778237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.778237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.778237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:09.778237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.778237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:09.778680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.779740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:09.780395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.780395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:09.852434 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:09.853128 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:09.853749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:09.854035 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:09.854385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.854697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:09.854697 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.854697 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:09.854697 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.854697 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.854697 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:09.854697 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.854697 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.856289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:09.856789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:14.494004 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:14.496273 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:14.530124 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:14.530350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.530909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:14.530909 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.530909 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:14.530909 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.530909 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.530909 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:14.530909 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.530909 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.531827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.532419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:14.643569 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:14.644558 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:14.644726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:14.644877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.645615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:14.645615 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.645615 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:14.645615 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.645615 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.645615 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:14.645615 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.645615 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.646432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.646883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:14.672609 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:14.673140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:14.673602 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:14.673789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.675078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:14.675078 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.675078 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:14.675078 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.675078 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.675078 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:14.675078 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.675078 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:14.675950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:14.676692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:14.676692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.676692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:14.676692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.676692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.676692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:14.676692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.676692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:14.677705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.678867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.679438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:14.679438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.679438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:14.774819 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:14.775398 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:14.776059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:14.776486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.777050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:14.777050 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.777050 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:14.777050 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.777050 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.777050 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:14.777050 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.777050 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:14.777845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:14.778375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:14.778375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.778375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:14.778375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.778375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.778375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:14.778375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.778375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:14.778615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.779088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:14.779580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.779580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:14.852987 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:14.853928 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:14.854669 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:14.854901 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:14.855134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.855939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:14.855939 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.855939 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:14.855939 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.855939 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.855939 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:14.855939 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.855939 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.856989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:14.857587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:19.495551 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:19.496834 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:19.524393 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:19.524826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.525686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:19.525686 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.525686 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:19.525686 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.525686 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.525686 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:19.525686 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.525686 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.526628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.527467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.650622 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.652576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.653294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:19.673036 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:19.673568 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:19.673997 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:19.674277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.674790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:19.674790 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.674790 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:19.674790 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.674790 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.674790 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:19.674790 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.674790 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:19.675704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:19.676512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:19.676512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.676512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:19.676512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.676512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.676512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:19.676512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.676512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:19.676835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.677487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.678200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:19.678200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.678200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:19.775662 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:19.776855 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:19.777543 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:19.777993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.778907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:19.778907 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.778907 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:19.778907 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.778907 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.778907 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:19.778907 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.778907 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:19.779842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:19.780175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:19.780175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.780175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:19.780175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.780175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.780175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:19.780175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.780175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:19.780949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.781296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:19.781799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.781799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:19.853729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:19.854311 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:19.854895 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:19.855364 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:19.855570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.856883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:19.856883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.856883 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:19.856883 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.856883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.856883 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:19.856883 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.856883 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.858174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:19.858709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:24.496188 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:24.498888 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:24.556498 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:24.556969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.558152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:24.558152 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.558152 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:24.558152 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.558152 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.558152 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:24.558152 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.558152 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.559443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.560569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:24.645709 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:24.647075 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:24.647451 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:24.647693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.648425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:24.648425 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.648425 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:24.648425 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.648425 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.648425 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:24.648425 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.648425 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.649723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.650661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:24.673624 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:24.674096 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:24.674568 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:24.674883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.675626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:24.675626 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.675626 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:24.675626 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.675626 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.675626 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:24.675626 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.675626 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:24.677002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.677811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:24.678587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:24.679444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:24.679444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:24.679444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:24.679444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:24.679444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.679444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:24.679444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.679444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.679444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:24.679444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.679444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:24.777014 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:24.777549 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:24.778317 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:24.778724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.779554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:24.779554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.779554 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:24.779554 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.779554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.779554 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:24.779554 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.779554 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:24.781062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:24.781803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:24.781803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.781803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:24.781803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.781803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.781803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:24.781803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.781803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:24.782696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.783471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:24.784263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.784263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:24.854169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:24.854421 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:24.854964 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:24.855293 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:24.855467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.856018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:24.856018 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.856018 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:24.856018 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.856018 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.856018 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:24.856018 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.856018 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.856900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:24.857346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:29.497029 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:29.498956 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:29.538734 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:29.538994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.539678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:29.539678 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.539678 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:29.539678 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.539678 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.539678 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:29.539678 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.539678 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.540956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.541394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:29.646680 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:29.648015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:29.648279 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:29.649025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.649025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:29.649025 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.649025 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:29.649025 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.649025 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.649025 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:29.649025 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.649025 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.652445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.652973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:29.677638 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:29.678270 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:29.678615 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:29.678796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.679546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:29.679546 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.679546 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:29.679546 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.679546 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.679546 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:29.679546 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.679546 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:29.680774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:29.681866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:29.681866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.681866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:29.681866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.681866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.681866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:29.681866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.681866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:29.682956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.683935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.685077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:29.685077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.685077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:29.777820 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:29.778342 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:29.779164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:29.779531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.780134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:29.780134 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.780134 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:29.780134 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.780134 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.780134 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:29.780134 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.780134 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:29.781169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.781951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:29.782729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:29.783588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:29.783588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:29.783588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.783588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:29.783588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.783588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.783588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:29.783588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.783588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:29.854630 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:29.854922 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.858288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:29.858990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:29.859834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:34.497700 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:34.499094 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:34.501039 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=471,neutron:UPDATE=2 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:28:34.533577 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:34.533911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.534413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:34.534413 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.534413 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:34.534413 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.534413 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.534413 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:34.534413 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.534413 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.535381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.536734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:34.647540 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:34.648668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:34.649518 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:34.649771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.650507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:34.650507 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.650507 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:34.650507 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.650507 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.650507 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:34.650507 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.650507 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.651636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.652518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:34.678374 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:34.679179 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:34.679584 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:34.679753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.680234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:34.680234 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.680234 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:34.680234 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.680234 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.680234 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:34.680234 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.680234 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:34.681090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:34.681825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:34.681825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.681825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:34.681825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.681825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.681825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:34.681825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.681825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:34.682158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.682852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.683648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:34.683648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.683648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:34.778559 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:34.779162 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:34.779610 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:34.779836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.780410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:34.780410 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.780410 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:34.780410 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.780410 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.780410 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:34.780410 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.780410 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:34.781812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:34.782442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:34.782442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.782442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:34.782442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.782442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.782442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:34.782442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.782442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:34.784064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.784703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:34.785296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.785296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:34.855173 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:34.855680 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:34.856957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:34.857794 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:34.857794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.858156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:34.858156 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.858156 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:34.858156 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.858156 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.858156 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:34.858156 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.858156 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.859290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:34.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:39.498286 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:39.499529 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:39.532922 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:39.534077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.535379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:39.535379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.535379 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:39.535379 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.535379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.535379 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:39.535379 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.535379 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.536574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.537123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:39.648146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:39.650891 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:39.650891 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:39.650891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.650891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:39.650891 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.650891 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:39.650891 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.650891 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.650891 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:39.650891 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.650891 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.652040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.652551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:39.679449 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:39.679937 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:39.680381 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:39.680548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.681971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:39.681971 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.681971 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:39.681971 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.681971 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.681971 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:39.681971 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.681971 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:39.683010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:39.683582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:39.683582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.683582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:39.683582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.683582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.683582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:39.683582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.683582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:39.683845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.684306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.684812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:39.684812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.684812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:39.779165 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:39.779694 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:39.780111 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:39.780371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.780894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:39.780894 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.780894 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:39.780894 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.780894 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.780894 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:39.780894 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.780894 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:39.782507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.784332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:39.785494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:39.786706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:39.786706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:39.786706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.786706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:39.786706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.786706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.786706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:39.786706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.786706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:39.855704 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:39.856298 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:39.856852 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:39.857238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:39.857484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.858475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:39.858475 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.858475 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:39.858475 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.858475 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.858475 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:39.858475 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.858475 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.860265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:39.861263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:44.498870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:44.500561 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:44.532054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:44.532424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.533169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:44.533169 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.533169 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:44.533169 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.533169 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.533169 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:44.533169 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.533169 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.534539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.535023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:44.648831 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:44.650285 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:44.650285 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:44.650285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.650521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:44.650521 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.650521 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:44.650521 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.650521 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.650521 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:44.650521 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.650521 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.651317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.651673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:44.680330 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:44.680978 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:44.681675 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:44.681908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:44.685015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.685461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:44.685751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.686095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:44.779979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:44.780634 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:44.781367 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:44.781841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.782707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:44.782707 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.782707 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:44.782707 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.782707 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.782707 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:44.782707 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.782707 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:44.783665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:44.784125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:44.784125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.784125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:44.784125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.784125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.784125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:44.784125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.784125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:44.787172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.787673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:44.788120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.788120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:44.857637 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:44.857637 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:44.857973 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:44.858482 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:44.858824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.860171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:44.860171 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.860171 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:44.860171 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.860171 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.860171 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:44.860171 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.860171 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.862033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:44.862556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:49.499729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:49.500917 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:49.535129 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:49.535522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.536200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:49.536200 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.536200 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:49.536200 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.536200 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.536200 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:49.536200 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.536200 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.537282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.538422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:49.649607 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:49.650693 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:49.650918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:49.651041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.651732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:49.651732 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.651732 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:49.651732 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.651732 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.651732 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:49.651732 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.651732 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.652924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.653933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:49.680938 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:49.681417 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:49.681950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:49.682230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.684868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:49.684868 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.684868 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:49.684868 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.684868 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.684868 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:49.684868 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.684868 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:49.686641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:49.687660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:49.687660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.687660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:49.687660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.687660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.687660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:49.687660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.687660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:49.688589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.689441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.690979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:49.690979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.690979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:49.780981 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:49.781465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:49.782083 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:49.782534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.783360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:49.783360 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.783360 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:49.783360 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.783360 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.783360 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:49.783360 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.783360 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:49.784541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:49.785899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:49.785899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.785899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:49.785899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.785899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.785899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:49.785899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.785899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:49.791767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.792292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:49.792816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.792816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:49.857610 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:49.857610 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:49.858576 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:49.858891 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:49.858891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.859578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:49.859578 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.859578 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:49.859578 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.859578 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.859578 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:49.859578 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.859578 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.860899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:49.861863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:54.500268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:54.502053 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:54.536279 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:54.536279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.536279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:54.536279 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.536279 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:54.536279 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.536279 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.536279 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:54.536279 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.536279 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.536623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.536946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:54.650415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:54.651977 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:54.652302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:54.652739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.653784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:54.653784 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.653784 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:54.653784 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.653784 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.653784 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:54.653784 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.653784 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.655864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.656256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:54.681496 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:54.682074 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:54.682840 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:54.683171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.684037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:54.684037 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.684037 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:54.684037 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.684037 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.684037 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:54.684037 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.684037 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:54.685491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:54.685956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:54.685956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.685956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:54.685956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.685956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.685956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:54.685956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.685956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:54.687106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.687583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.688189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:54.688189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.688189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:54.781833 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:54.782186 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:54.782604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:54.782964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.783903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:54.783903 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.783903 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:54.783903 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.783903 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.783903 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:54.783903 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.783903 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:54.785113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:54.786006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:54.786006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.786006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:54.786006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.786006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.786006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:54.786006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.786006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:54.786404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.787165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:54.787989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.787989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:54.857749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:54.857950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:54.858430 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:54.858636 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:54.858812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.859360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:54.859360 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.859360 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:54.859360 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.859360 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.859360 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:54.859360 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.859360 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.861162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:54.861685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:59.500844 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:59.502614 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:59.540789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:59.541087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.541838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:28:59.541838 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.541838 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:59.541838 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.541838 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.541838 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:59.541838 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.541838 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.543016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.543376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:59.651153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:59.652533 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:59.652671 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:59.652797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.653486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:59.653486 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.653486 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:59.653486 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.653486 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.653486 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:59.653486 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.653486 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.654693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.655067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:59.681991 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:59.682491 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:28:59.682991 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:59.683224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.683802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:59.683802 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.683802 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:59.683802 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.683802 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.683802 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:59.683802 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.683802 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:59.685198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:59.685494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:59.685494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.685494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:59.685494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.685494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.685494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:59.685494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.685494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:28:59.687175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.687494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.687822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:59.687822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.687822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:59.782318 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:59.782702 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:59.783433 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:59.784063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.784883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:59.784883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.784883 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:59.784883 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.784883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.784883 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:59.784883 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.784883 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:28:59.786562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:59.787491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:28:59.787491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.787491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:28:59.787491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.787491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.787491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:28:59.787491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.787491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:28:59.788103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.788835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:59.789701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.789701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:28:59.858273 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:28:59.858719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:28:59.859085 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:59.859500 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:28:59.859794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.860580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:28:59.860580 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.860580 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:28:59.860580 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.860580 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.860580 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:28:59.860580 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.860580 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.862083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:28:59.863078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:04.501528 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:04.502732 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:04.539009 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:04.539275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.539846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:04.539846 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.539846 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:04.539846 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.539846 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.539846 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:04.539846 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.539846 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.540929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.541313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:04.651752 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:04.652699 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:04.653884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:04.653884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.653884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:04.653884 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.653884 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:04.653884 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.653884 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.653884 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:04.653884 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.653884 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.654836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.655985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:04.682505 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:04.683102 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:04.683716 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:04.683991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.684655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:04.684655 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.684655 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:04.684655 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.684655 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.684655 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:04.684655 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.684655 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:04.685709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:04.686243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:04.686243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.686243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:04.686243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.686243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.686243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:04.686243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.686243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:04.686955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.687981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.688909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:04.688909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.688909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:04.783015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:04.783692 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:04.784388 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:04.784824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.785903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:04.785903 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.785903 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:04.785903 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.785903 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.785903 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:04.785903 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.785903 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:04.786781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:04.787243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:04.787243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.787243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:04.787243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.787243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.787243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:04.787243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.787243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:04.787800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.788277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:04.788760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.788760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:04.858929 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:04.859452 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:04.859977 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:04.860466 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:04.860682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.861460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:04.861460 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.861460 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:04.861460 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.861460 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.861460 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:04.861460 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.861460 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.862878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:04.863696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:09.502507 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:09.503163 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:09.536794 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:09.537097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.537843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:09.537843 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.537843 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:09.537843 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.537843 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.537843 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:09.537843 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.537843 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.538737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.539192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:09.652644 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:09.657672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:09.657672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:09.657672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.657912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:09.657912 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.657912 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:09.657912 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.657912 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.657912 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:09.657912 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.657912 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.659757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:09.683124 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:09.683556 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:09.683983 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:09.684746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.685421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:09.685421 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.685421 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:09.685421 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.685421 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.685421 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:09.685421 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.685421 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:09.686760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:09.687229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:09.687229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.687229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:09.687229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.687229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.687229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:09.687229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.687229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:09.689619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.690057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.690538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:09.690538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.690538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:09.783896 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:09.784552 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:09.785138 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:09.786565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.786565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:09.786565 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.786565 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:09.786565 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.786565 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.786565 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:09.786565 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.786565 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:09.787487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.789107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:09.789850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:09.790622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:09.790622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:09.790622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.790622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:09.790622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.790622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.790622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:09.790622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.790622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:09.859511 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:09.859951 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:09.861653 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:09.861653 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:09.861653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.861849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:09.861849 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.861849 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:09.861849 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.861849 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.861849 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:09.861849 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.861849 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.862832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:09.863605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:14.502900 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:14.504635 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:14.541417 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:14.541744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.542723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:14.542723 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.542723 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:14.542723 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.542723 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.542723 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:14.542723 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.542723 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.543626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.544856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:14.655939 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:14.657236 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:14.657497 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:14.657780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.658728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:14.658728 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.658728 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:14.658728 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.658728 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.658728 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:14.658728 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.658728 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.660532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.661397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:14.683460 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:14.684062 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:14.684448 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:14.684671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.685320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:14.685320 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.685320 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:14.685320 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.685320 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.685320 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:14.685320 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.685320 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:14.686108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.687023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:14.687855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:14.688790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:14.688790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:14.688790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:14.688790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:14.688790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.688790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:14.688790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.688790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.688790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:14.688790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.688790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:14.784798 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:14.785627 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:14.786399 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:14.786795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.787736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:14.787736 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.787736 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:14.787736 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.787736 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.787736 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:14.787736 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.787736 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:14.788904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:14.789398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:14.789398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.789398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:14.789398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.789398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.789398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:14.789398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.789398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:14.790879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.791591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:14.792943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.792943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:14.860491 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:14.860974 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:14.861519 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:14.861798 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:14.861961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.862888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:14.862888 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.862888 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:14.862888 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.862888 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.862888 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:14.862888 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.862888 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.864030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:14.864528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:19.503600 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:19.506155 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:19.549204 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:19.549826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.550945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:19.550945 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.550945 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:19.550945 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.550945 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.550945 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:19.550945 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.550945 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.551763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.552313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:19.657703 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:19.657940 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:19.658280 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:19.658504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.661571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:19.661571 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.661571 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:19.661571 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.661571 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.661571 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:19.661571 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.661571 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.663077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.663825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:19.684025 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:19.684365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:19.684893 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:19.685206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.685942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:19.685942 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.685942 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:19.685942 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.685942 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.685942 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:19.685942 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.685942 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:19.687618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:19.688856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:19.688856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.688856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:19.688856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.688856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.688856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:19.688856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.688856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:19.691335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.692104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.692955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:19.692955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.692955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:19.785862 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:19.786483 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:19.787208 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:19.787530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.788252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:19.788252 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.788252 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:19.788252 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.788252 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.788252 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:19.788252 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.788252 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:19.790076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:19.790575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:19.790575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.790575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:19.790575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.790575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.790575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:19.790575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.790575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:19.791511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.791969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:19.792485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.792485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:19.861187 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:19.861784 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:19.862392 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:19.862613 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:19.862732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.863304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:19.863304 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.863304 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:19.863304 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.863304 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.863304 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:19.863304 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.863304 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.864459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:19.865362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:24.504354 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:24.505371 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:24.538120 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:24.538390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.539340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:24.539340 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.539340 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:24.539340 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.539340 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.539340 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:24.539340 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.539340 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.540422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.541048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:24.657386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:24.658436 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:24.658703 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:24.658882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.659961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:24.659961 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.659961 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:24.659961 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.659961 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.659961 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:24.659961 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.659961 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.661605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.662457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:24.684442 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:24.684864 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:24.685340 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:24.685569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.686036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:24.686036 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.686036 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:24.686036 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.686036 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.686036 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:24.686036 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.686036 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:24.686965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.688114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:24.689159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:24.690438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:24.690438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:24.690438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:24.690438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:24.690438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.690438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:24.690438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.690438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.690438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:24.690438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.690438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:24.786801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:24.787644 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:24.788388 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:24.788793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.789575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:24.789575 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.789575 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:24.789575 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.789575 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.789575 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:24.789575 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.789575 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:24.790867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:24.791425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:24.791425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.791425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:24.791425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.791425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.791425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:24.791425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.791425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:24.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.792788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:24.793296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.793296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:24.861797 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:24.862250 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:24.862932 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:24.863334 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:24.863602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.864201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:24.864201 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.864201 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:24.864201 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.864201 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.864201 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:24.864201 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.864201 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.866207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:24.867036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:29.504996 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:29.506684 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:29.543370 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:29.543580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.544158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:29.544158 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.544158 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:29.544158 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.544158 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.544158 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:29.544158 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.544158 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.545371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.546119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:29.658032 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:29.661671 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:29.661970 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:29.662086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.662973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:29.662973 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.662973 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:29.662973 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.662973 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.662973 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:29.662973 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.662973 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.664688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.665027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:29.684983 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:29.685404 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:29.685800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:29.686007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.686549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:29.686549 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.686549 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:29.686549 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.686549 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.686549 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:29.686549 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.686549 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:29.687138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:29.687498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:29.687498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.687498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:29.687498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.687498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.687498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:29.687498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.687498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:29.687857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.688154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.688499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:29.688499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.688499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:29.787766 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:29.788326 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:29.789101 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:29.789631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.790456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:29.790456 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.790456 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:29.790456 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.790456 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.790456 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:29.790456 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.790456 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:29.791341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:29.791758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:29.791758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.791758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:29.791758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.791758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.791758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:29.791758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.791758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:29.792565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.793124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:29.793595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.793595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:29.862410 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:29.863060 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:29.863877 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:29.864306 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:29.864538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.865457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:29.865457 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.865457 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:29.865457 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.865457 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.865457 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:29.865457 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.865457 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.867244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:29.868296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:34.506185 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:34.507718 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:34.513767 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=241 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:29:34.548200 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:34.548438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.549107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:34.549107 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.549107 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:34.549107 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.549107 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.549107 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:34.549107 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.549107 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.550274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.551164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:34.658807 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:34.660667 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:34.660667 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:34.660667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.661039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:34.661039 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.661039 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:34.661039 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.661039 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.661039 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:34.661039 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.661039 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.662958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.663865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:34.685514 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:34.687328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:34.687835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:34.688106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.688701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:34.688701 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.688701 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:34.688701 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.688701 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.688701 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:34.688701 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.688701 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:34.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.691748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:34.692049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:34.692401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:34.692401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:34.692401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:34.692401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:34.692401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.692401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:34.692401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.692401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.692401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:34.692401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.692401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:34.788576 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:34.788992 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:34.789546 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:34.789762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.790286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:34.790286 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.790286 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:34.790286 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.790286 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.790286 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:34.790286 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.790286 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:34.791160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:34.792003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:34.792003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.792003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:34.792003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.792003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.792003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:34.792003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.792003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:34.792354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.792950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:34.793464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.793464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:34.863069 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:34.863375 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:34.864051 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:34.864525 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:34.864838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.865707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:34.865707 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.865707 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:34.865707 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.865707 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.865707 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:34.865707 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.865707 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.866718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:34.867548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:39.506949 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:39.508542 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:39.547264 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:39.547531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.548065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:39.548065 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.548065 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:39.548065 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.548065 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.548065 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:39.548065 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.548065 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.548881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.549922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:39.659291 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:39.660025 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:39.660241 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:39.660362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.660895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:39.660895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.660895 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:39.660895 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.660895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.660895 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:39.660895 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.660895 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.661645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.662569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:39.687354 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:39.687589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:39.687897 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:39.688058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.688508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:39.688508 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.688508 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:39.688508 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.688508 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.688508 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:39.688508 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.688508 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:39.689044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.690060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:39.690630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:39.691362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:39.691362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:39.691362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:39.691362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:39.691362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.691362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:39.691362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.691362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.691362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:39.691362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.691362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:39.789246 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:39.789699 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:39.790178 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:39.790677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.791121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:39.791121 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.791121 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:39.791121 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.791121 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.791121 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:39.791121 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.791121 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:39.792503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:39.793612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:39.793612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.793612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:39.793612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.793612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.793612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:39.793612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.793612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:39.794258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.795420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:39.796715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.796715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:39.863611 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:39.864058 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:39.864534 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:39.864821 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:39.865009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.865585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:39.865585 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.865585 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:39.865585 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.865585 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.865585 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:39.865585 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.865585 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.866773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:39.867517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:44.507621 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:44.508677 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:44.538494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:44.538684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.539344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:44.539344 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.539344 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:44.539344 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.539344 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.539344 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:44.539344 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.539344 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.540267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.540621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:44.661082 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:44.661082 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:44.661082 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:44.661082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.661532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:44.661532 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.661532 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:44.661532 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.661532 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.661532 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:44.661532 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.661532 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.662681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.663171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:44.687944 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:44.688347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:44.689292 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:44.689655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.690439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:44.690439 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.690439 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:44.690439 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.690439 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.690439 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:44.690439 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.690439 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:44.691778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:44.692839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:44.692839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.692839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:44.692839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.692839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.692839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:44.692839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.692839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:44.694124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.694921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.697347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:44.697347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.697347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:44.789853 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:44.790615 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:44.791306 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:44.791609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.792290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:44.792290 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.792290 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:44.792290 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.792290 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.792290 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:44.792290 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.792290 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:44.793531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.795132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:44.797435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:44.799040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:44.799040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:44.799040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.799040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:44.799040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.799040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.799040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:44.799040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.799040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:44.864113 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:44.864555 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:44.865042 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:44.865239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:44.865373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.866438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:44.866438 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.866438 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:44.866438 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.866438 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.866438 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:44.866438 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.866438 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:44.868342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:49.508330 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:49.510017 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:49.547418 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:49.547764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.548618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:49.548618 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.548618 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:49.548618 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.548618 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.548618 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:49.548618 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.548618 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.549527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.550007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:49.660339 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:49.662920 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:49.663149 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:49.663323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.665793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:49.666836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.667771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.667771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:49.667771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.667771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:49.688515 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:49.688977 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:49.689402 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:49.689660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.690195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:49.690195 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.690195 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:49.690195 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.690195 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.690195 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:49.690195 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.690195 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:49.691115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:49.691616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:49.691616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.691616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:49.691616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.691616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.691616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:49.691616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.691616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:49.692073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.692565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.693094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:49.693094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.693094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:49.790733 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:49.791088 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:49.791827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:49.792239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.793111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:49.793111 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.793111 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:49.793111 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.793111 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.793111 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:49.793111 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.793111 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:49.794456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:49.794917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:49.794917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.794917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:49.794917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.794917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.794917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:49.794917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.794917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:49.796000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.796484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:49.796996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.796996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:49.864731 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:49.865555 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:49.866159 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:49.866438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:49.866618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.867208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:49.867208 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.867208 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:49.867208 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.867208 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.867208 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:49.867208 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.867208 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.868845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:49.869640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:54.508972 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:54.510301 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:54.542680 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:54.542995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.543811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:54.543811 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.543811 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:54.543811 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.543811 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.543811 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:54.543811 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.543811 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.544724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.545709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:54.662180 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:54.663796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:54.664235 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:54.664563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:54.668110 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.668110 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:54.668110 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.668110 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.668110 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:54.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.668110 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.669359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.670183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:54.689362 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:54.689982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:54.690891 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:54.691577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.692176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:54.692176 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.692176 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:54.692176 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.692176 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.692176 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:54.692176 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.692176 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:54.693381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:54.693906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:54.693906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.693906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:54.693906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.693906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.693906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:54.693906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.693906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:54.695716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.697551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.698698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:54.698698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.698698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:54.791238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:54.791750 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:54.792431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:54.792853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.793759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:54.793759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.793759 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:54.793759 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.793759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.793759 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:54.793759 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.793759 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:54.795195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:54.795653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:54.795653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.795653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:54.795653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.795653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.795653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:54.795653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.795653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:54.796873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.797760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:54.798859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.798859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:54.865758 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:54.866434 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:54.867170 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:54.867653 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:54.867891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.868957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:54.868957 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.868957 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:54.868957 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.868957 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.868957 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:54.868957 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.868957 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.870442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:54.871337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:59.509572 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:59.510781 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:59.541017 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:59.541470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.542489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:29:59.542489 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.542489 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:59.542489 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.542489 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.542489 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:59.542489 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.542489 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.543372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.544248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:59.662951 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:59.664241 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:59.664576 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:59.664840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.665735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:59.665735 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.665735 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:59.665735 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.665735 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.665735 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:59.665735 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.665735 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.666972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.667381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:59.690008 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:59.690464 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:29:59.690940 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:59.691337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.692252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:59.692252 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.692252 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:59.692252 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.692252 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.692252 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:59.692252 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.692252 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:59.693371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:59.693931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:59.693931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.693931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:59.693931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.693931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.693931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:59.693931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.693931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:29:59.694890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.695545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.696318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:59.696318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.696318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:59.791811 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:59.792237 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:59.792918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:59.793369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.794140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:59.794140 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.794140 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:59.794140 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.794140 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.794140 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:59.794140 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.794140 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:29:59.795039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:59.795766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:29:59.795766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.795766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:29:59.795766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.795766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.795766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:29:59.795766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.795766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:29:59.796189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:59.797750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.797750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:29:59.866125 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:29:59.866751 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:29:59.867397 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:59.867698 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:29:59.867854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.868689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:29:59.868689 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.868689 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:29:59.868689 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.868689 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.868689 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:29:59.868689 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.868689 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.869656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:29:59.870444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:04.510651 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:04.511952 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:04.549336 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:04.549935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.551129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:04.551129 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.551129 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:04.551129 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.551129 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.551129 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:04.551129 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.551129 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.552159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.553556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:04.663692 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:04.664749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:04.665046 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:04.665671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.666163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:04.666163 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.666163 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:04.666163 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.666163 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.666163 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:04.666163 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.666163 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.667772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.669021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:04.690796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:04.691282 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:04.691751 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:04.692008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.692615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:04.692615 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.692615 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:04.692615 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.692615 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.692615 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:04.692615 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.692615 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:04.693504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:04.694234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:04.694234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.694234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:04.694234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.694234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.694234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:04.694234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.694234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:04.694866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.695381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.695859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:04.695859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.695859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:04.792343 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:04.792842 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:04.793570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:04.794058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.794993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:04.794993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.794993 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:04.794993 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.794993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.794993 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:04.794993 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.794993 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:04.795870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:04.796652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:04.796652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.796652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:04.796652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.796652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.796652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:04.796652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.796652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:04.797004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.797832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:04.798823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.798823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:04.866796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:04.867434 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:04.868181 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:04.869381 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:04.869381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.870026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:04.870026 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.870026 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:04.870026 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.870026 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.870026 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:04.870026 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.870026 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.871533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:04.872307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:09.511484 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:09.513476 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:09.549953 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:09.550151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.550711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:09.550711 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.550711 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:09.550711 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.550711 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.550711 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:09.550711 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.550711 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.551645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.552042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:09.664524 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:09.665523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:09.665910 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:09.666039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.666716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:09.666716 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.666716 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:09.666716 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.666716 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.666716 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:09.666716 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.666716 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.667998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.668953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:09.691477 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:09.691876 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:09.692384 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:09.692675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.693416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:09.693416 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.693416 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:09.693416 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.693416 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.693416 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:09.693416 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.693416 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:09.694312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:09.695117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:09.695117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.695117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:09.695117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.695117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.695117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:09.695117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.695117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:09.695538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.696139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.696638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:09.696638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.696638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:09.793067 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:09.793530 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:09.794117 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:09.794562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.795352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:09.795352 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.795352 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:09.795352 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.795352 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.795352 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:09.795352 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.795352 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:09.796559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:09.797389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:09.797389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.797389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:09.797389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.797389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.797389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:09.797389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.797389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:09.798116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.798980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:09.799989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.799989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:09.867679 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:09.868543 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:09.868957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:09.869832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:09.869832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.870359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:09.870359 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.870359 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:09.870359 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.870359 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.870359 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:09.870359 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.870359 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:09.873030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:14.512250 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:14.513847 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:14.545780 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:14.545999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.546631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:14.546631 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.546631 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:14.546631 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.546631 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.546631 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:14.546631 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.546631 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.547876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.548444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:14.665135 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:14.666424 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:14.666757 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:14.666996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.667919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:14.667919 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.667919 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:14.667919 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.667919 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.667919 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:14.667919 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.667919 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.669261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.669724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:14.692033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:14.692494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:14.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:14.693307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.694162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:14.694162 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.694162 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:14.694162 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.694162 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.694162 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:14.694162 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.694162 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:14.695382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:14.695907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:14.695907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.695907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:14.695907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.695907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.695907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:14.695907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.695907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:14.696448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.696943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.697494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:14.697494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.697494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:14.801184 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:14.801992 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:14.802956 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:14.803200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.803919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:14.803919 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.803919 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:14.803919 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.803919 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.803919 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:14.803919 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.803919 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:14.804741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:14.805287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:14.805287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.805287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:14.805287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.805287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.805287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:14.805287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.805287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:14.805653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.806449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:14.807274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.807274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:14.868391 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:14.869166 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:14.869885 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:14.870495 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:14.870797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.871748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:14.871748 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.871748 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:14.871748 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.871748 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.871748 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:14.871748 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.871748 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.873487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:14.873928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:19.513037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:19.515047 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:19.551204 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:19.551435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.552147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:19.552147 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.552147 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:19.552147 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.552147 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.552147 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:19.552147 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.552147 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.552904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.553764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:19.665932 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:19.667352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:19.667717 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:19.667976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.669108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:19.669108 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.669108 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:19.669108 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.669108 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.669108 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:19.669108 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.669108 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.670801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.672441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:19.692685 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:19.693308 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:19.693937 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:19.694172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.694983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:19.694983 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.694983 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:19.694983 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.694983 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.694983 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:19.694983 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.694983 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:19.696540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:19.697077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:19.697077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.697077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:19.697077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.697077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.697077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:19.697077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.697077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:19.698230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.698682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.699140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:19.699140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.699140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:19.801907 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:19.802450 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:19.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:19.803608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.804540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:19.804540 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.804540 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:19.804540 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.804540 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.804540 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:19.804540 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.804540 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:19.805886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:19.806660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:19.806660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.806660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:19.806660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.806660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.806660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:19.806660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.806660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:19.807351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.808114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:19.809013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.809013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:19.869076 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:19.869630 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:19.870155 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:19.870575 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:19.870813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.871604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:19.871604 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.871604 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:19.871604 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.871604 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.871604 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:19.871604 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.871604 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.873320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:19.873891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:24.514523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:24.515919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:24.548606 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:24.548606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.548813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:24.548813 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.548813 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:24.548813 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.548813 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.548813 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:24.548813 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.548813 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.549726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:24.666638 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:24.668814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:24.668814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:24.668814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.669253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:24.669253 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.669253 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:24.669253 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.669253 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.669253 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:24.669253 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.669253 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.671403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.671838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:24.693547 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:24.694133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:24.694687 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:24.695187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.696621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:24.696621 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.696621 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:24.696621 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.696621 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.696621 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:24.696621 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.696621 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:24.697392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:24.697862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:24.697862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.697862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:24.697862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.697862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.697862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:24.697862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.697862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:24.698133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.698689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.699193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:24.699193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.699193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:24.802870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:24.803792 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:24.804416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:24.805588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.806302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:24.806302 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.806302 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:24.806302 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.806302 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.806302 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:24.806302 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.806302 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:24.807009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:24.807859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:24.807859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.807859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:24.807859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.807859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.807859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:24.807859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.807859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:24.808177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.809027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:24.810581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.810581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:24.871515 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:24.872485 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:24.873160 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:24.873645 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:24.875104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.877404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:24.878388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:24.879341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:24.879341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:24.879341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:24.879341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:29.515409 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:29.517237 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:29.550257 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:29.550565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.551254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:29.551254 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.551254 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:29.551254 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.551254 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.551254 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:29.551254 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.551254 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.552169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.552966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:29.667357 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:29.668735 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:29.669504 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:29.669592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.670729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:29.670729 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.670729 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:29.670729 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.670729 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.670729 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:29.670729 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.670729 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.672071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.672599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:29.694139 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:29.694618 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:29.695465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:29.695695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.696391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:29.696391 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.696391 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:29.696391 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.696391 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.696391 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:29.696391 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.696391 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:29.697739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:29.699460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:29.699460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.699460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:29.699460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.699460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.699460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:29.699460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.699460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:29.700265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.701400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.702705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:29.702705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.702705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:29.803493 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:29.803922 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:29.804543 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:29.804750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.805239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:29.805239 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.805239 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:29.805239 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.805239 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.805239 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:29.805239 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.805239 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:29.805872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:29.806577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:29.806577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.806577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:29.806577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.806577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.806577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:29.806577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.806577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:29.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.807529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:29.808125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.808125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:29.872321 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:29.873475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:29.874005 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:29.874234 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:29.874341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.875306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:29.875306 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.875306 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:29.875306 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.875306 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.875306 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:29.875306 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.875306 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.876507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:29.877062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:34.515991 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:34.517128 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:34.548178 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=256 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:30:34.552106 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:34.552278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.552844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:34.552844 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.552844 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:34.552844 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.552844 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.552844 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:34.552844 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.552844 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.553992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.554381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:34.668366 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:34.669155 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:34.669363 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:34.669656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.670175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:34.670175 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.670175 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:34.670175 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.670175 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.670175 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:34.670175 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.670175 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.671967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.672855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:34.694485 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:34.695355 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:34.695966 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:34.696143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.696657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:34.696657 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.696657 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:34.696657 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.696657 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.696657 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:34.696657 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.696657 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:34.697274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.698277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:34.699008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:34.699874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:34.699874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:34.699874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:34.699874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:34.699874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.699874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:34.699874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.699874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.699874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:34.699874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.699874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:34.804142 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:34.804572 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:34.805462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:34.805835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.806784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:34.806784 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.806784 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:34.806784 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.806784 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.806784 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:34.806784 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.806784 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:34.807812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:34.809191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:34.809191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.809191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:34.809191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.809191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.809191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:34.809191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.809191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:34.809800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:34.812137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.812137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:34.873534 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:34.873937 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:34.874354 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:34.874721 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:34.874983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.875924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:34.875924 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.875924 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:34.875924 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.875924 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.875924 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:34.875924 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.875924 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:34.878440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:39.516524 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:39.517651 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:39.553474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:39.553474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.553474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:39.553474 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.553474 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:39.553474 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.553474 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.553474 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:39.553474 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.553474 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.554203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.554956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:39.668496 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:39.669950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:39.670465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:39.670785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.671865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:39.671865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.671865 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:39.671865 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.671865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.671865 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:39.671865 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.671865 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.673578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.674714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:39.695409 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:39.695853 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:39.696351 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:39.696560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.697281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:39.697281 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.697281 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:39.697281 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.697281 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.697281 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:39.697281 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.697281 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:39.698646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:39.699144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:39.699144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.699144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:39.699144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.699144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.699144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:39.699144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.699144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:39.700354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.700946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.701429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:39.701429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.701429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:39.804828 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:39.805649 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:39.805996 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:39.806168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.807489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:39.807489 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.807489 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:39.807489 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.807489 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.807489 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:39.807489 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.807489 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:39.807855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:39.808373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:39.808373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.808373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:39.808373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.808373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.808373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:39.808373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.808373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:39.809371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.810000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:39.810547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.810547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:39.898715 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:39.899854 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:39.900950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:39.901595 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:39.901595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.901795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:39.901795 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.901795 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:39.901795 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.901795 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.901795 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:39.901795 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.901795 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.902681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:39.903061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:44.517475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:44.519105 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:44.553032 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:44.553474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.554162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:44.554162 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.554162 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:44.554162 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.554162 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.554162 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:44.554162 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.554162 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.555241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.555873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:44.669078 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:44.670339 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:44.670836 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:44.671080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.671964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:44.671964 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.671964 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:44.671964 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.671964 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.671964 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:44.671964 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.671964 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.673734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.674091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:44.695801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:44.696507 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:44.697612 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:44.698038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.699087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:44.699087 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.699087 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:44.699087 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.699087 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.699087 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:44.699087 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.699087 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:44.699960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:44.700312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:44.700312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.700312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:44.700312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.700312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.700312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:44.700312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.700312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:44.700736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.701039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.701448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:44.701448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.701448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:44.805700 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:44.806176 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:44.807185 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:44.807645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.808555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:44.808555 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.808555 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:44.808555 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.808555 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.808555 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:44.808555 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.808555 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:44.809690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.810621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:44.811615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:44.812467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:44.812467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:44.812467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.812467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:44.812467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.812467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.812467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:44.812467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.812467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:44.899895 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:44.900426 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:44.901934 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:44.901934 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:44.901934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.903637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:44.903637 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.903637 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:44.903637 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.903637 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.903637 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:44.903637 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.903637 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.904034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:44.904852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:49.517971 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:49.520119 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:49.552681 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:49.553061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.553968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:49.553968 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.553968 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:49.553968 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.553968 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.553968 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:49.553968 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.553968 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.554884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.555757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:49.669716 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:49.671098 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:49.671513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:49.671773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.672683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:49.672683 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.672683 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:49.672683 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.672683 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.672683 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:49.672683 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.672683 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.674677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.675572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:49.696537 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:49.697020 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:49.698288 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:49.698288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.698492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:49.698492 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.698492 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:49.698492 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.698492 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.698492 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:49.698492 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.698492 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:49.699265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:49.699638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:49.699638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.699638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:49.699638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.699638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.699638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:49.699638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.699638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:49.700195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.700515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:49.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.700855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:49.806329 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:49.806807 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:49.807326 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:49.807550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.807995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:49.807995 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.807995 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:49.807995 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.807995 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.807995 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:49.807995 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.807995 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:49.808642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.809370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:49.810373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:49.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:49.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:49.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:49.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:49.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.811310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:49.900640 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:49.901490 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:49.902076 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:49.902313 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:49.902562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.903020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:49.903020 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.903020 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:49.903020 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.903020 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.903020 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:49.903020 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.903020 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.904578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:49.904951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:54.518542 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:54.520619 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:54.555298 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:54.556444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.556763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:54.556763 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.556763 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:54.556763 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.556763 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.556763 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:54.556763 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.556763 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.558075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.558761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:54.670430 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:54.671824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:54.673919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:54.673919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.674252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:54.674252 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.674252 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:54.674252 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.674252 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.674252 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:54.674252 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.674252 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.675462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.676446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:54.696962 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:54.697566 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:54.698265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:54.698767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.700350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:54.700350 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.700350 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:54.700350 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.700350 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.700350 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:54.700350 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.700350 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:54.701858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:54.702727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:54.702727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.702727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:54.702727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.702727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.702727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:54.702727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.702727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:54.703788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.704597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.705086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:54.705086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.705086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:54.807054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:54.807686 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:54.809020 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:54.809020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.809316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:54.809316 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.809316 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:54.809316 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.809316 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.809316 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:54.809316 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.809316 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:54.810536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:54.811332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:54.811332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.811332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:54.811332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.811332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.811332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:54.811332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.811332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:54.812026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.812753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:54.813284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.813284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:54.901373 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:54.902628 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:54.903467 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:54.903891 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:54.904116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.905010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:54.905010 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.905010 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:54.905010 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.905010 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.905010 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:54.905010 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.905010 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.906944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:54.907761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:59.519389 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:59.520378 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:59.549810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:59.549994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.550585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:30:59.550585 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.550585 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:59.550585 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.550585 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.550585 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:59.550585 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.550585 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.551504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.552153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:59.671087 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:59.672493 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:59.672778 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:59.673053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.674163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:59.674163 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.674163 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:59.674163 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.674163 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.674163 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:59.674163 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.674163 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.675936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.676826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:59.697571 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:59.697980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:30:59.698445 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:59.698643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.699397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:59.699397 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.699397 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:59.699397 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.699397 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.699397 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:59.699397 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.699397 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:59.700396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:59.700702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:59.700702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.700702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:59.700702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.700702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.700702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:59.700702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.700702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:30:59.701877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.702172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.702536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:59.702536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.702536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:59.807681 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:59.808315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:59.808789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:59.809187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.809873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:30:59.809873 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.809873 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:59.809873 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.809873 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.809873 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:59.809873 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.809873 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:30:59.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:30:59.811502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:59.811502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:30:59.811502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.811502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:30:59.811502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.811502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.811502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:30:59.811502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.811502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:30:59.812126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.812929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:59.813853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.813853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:30:59.902721 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:30:59.903749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:30:59.904538 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:59.904928 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:30:59.905155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.906205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:30:59.906205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.906205 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:30:59.906205 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.906205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.906205 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:30:59.906205 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.906205 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.907385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:30:59.908008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:04.519993 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:04.521464 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:04.556693 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:04.556996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.557856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:04.557856 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.557856 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:04.557856 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.557856 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.557856 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:04.557856 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.557856 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.559095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.559637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:04.671750 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:04.673161 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:04.673439 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:04.673650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.674298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:04.674298 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.674298 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:04.674298 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.674298 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.674298 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:04.674298 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.674298 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.675671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.676477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:04.698098 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:04.698497 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:04.700570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:04.700570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.700771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:04.700771 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.700771 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:04.700771 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.700771 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.700771 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:04.700771 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.700771 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:04.702151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:04.703411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:04.703411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.703411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:04.703411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.703411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.703411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:04.703411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.703411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:04.703771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.704898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.706156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:04.706156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.706156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:04.808669 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:04.809686 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:04.810457 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:04.810836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.812199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:31:04.812199 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.812199 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:04.812199 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.812199 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.812199 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:04.812199 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.812199 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:04.814023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:04.814464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:04.814464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:04.814464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.814464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:04.814464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.814464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.814464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:04.814464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.814464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:04.816062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.816537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:04.817030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.817030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:04.903887 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:04.904473 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:04.905181 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:04.905627 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:04.905915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.906805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:04.906805 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.906805 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:04.906805 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.906805 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.906805 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:04.906805 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.906805 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.907917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:04.908467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:09.520642 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:09.521738 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:09.555155 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:09.555427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.556105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:09.556105 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.556105 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:09.556105 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.556105 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.556105 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:09.556105 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.556105 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.557308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.557732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:09.672357 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:09.673448 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:09.673694 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:09.673861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.674539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:09.674539 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.674539 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:09.674539 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.674539 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.674539 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:09.674539 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.674539 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.675892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.676447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:09.698613 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:09.699138 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:09.699601 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:09.699802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.700459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:09.700459 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.700459 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:09.700459 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.700459 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.700459 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:09.700459 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.700459 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:09.701748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:09.702291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:09.702291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.702291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:09.702291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.702291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.702291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:09.702291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.702291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:09.704533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.705045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.705571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:09.705571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.705571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:09.809655 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:09.810105 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:09.810566 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:09.810821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.811460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:09.811460 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.811460 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:09.811460 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.811460 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.811460 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:09.811460 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.811460 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:09.812858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:09.813530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:09.813530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.813530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:09.813530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.813530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.813530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:09.813530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.813530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:09.814589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.815084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:09.815578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.815578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:09.904782 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:09.905380 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:09.905768 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:09.905993 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:09.906142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.906829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:09.906829 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.906829 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:09.906829 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.906829 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.906829 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:09.906829 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.906829 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.908913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:09.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:14.521300 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:14.522882 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:14.566623 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:14.566877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.567740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:14.567740 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.567740 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:14.567740 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.567740 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.567740 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:14.567740 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.567740 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.569171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.570025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:14.672979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:14.674360 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:14.674701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:14.674940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.675914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:14.675914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.675914 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:14.675914 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.675914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.675914 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:14.675914 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.675914 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.677801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.678773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:14.699070 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:14.699514 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:14.700019 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:14.700340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.700996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:14.700996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.700996 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:14.700996 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.700996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.700996 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:14.700996 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.700996 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:14.702152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:14.702561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:14.702561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.702561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:14.702561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.702561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.702561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:14.702561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.702561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:14.703718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.704100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.704681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:14.704681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.704681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:14.810481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:14.810857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:14.811646 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:14.812037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.812835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:14.812835 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.812835 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:14.812835 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.812835 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.812835 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:14.812835 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.812835 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:14.814087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:14.815100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:14.815100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.815100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:14.815100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.815100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.815100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:14.815100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.815100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:14.815558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.816349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:14.817051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.817051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:14.905545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:14.905991 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:14.906502 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:14.906835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:14.907048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.907784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:14.907784 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.907784 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:14.907784 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.907784 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.907784 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:14.907784 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.907784 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.908628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:14.909092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:19.521925 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:19.523621 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:19.557138 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:19.557588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.558547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:19.558547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.558547 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:19.558547 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.558547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.558547 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:19.558547 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.558547 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.559712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.560789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:19.673711 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:19.675052 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:19.675466 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:19.675684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.676707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:19.676707 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.676707 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:19.676707 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.676707 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.676707 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:19.676707 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.676707 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.678567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.679369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:19.699620 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:19.699973 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:19.700551 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:19.700849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.701564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:19.701564 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.701564 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:19.701564 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.701564 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.701564 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:19.701564 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.701564 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:19.702812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.704514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:19.705251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:19.706048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:19.706048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:19.706048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:19.706048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:19.706048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.706048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:19.706048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.706048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.706048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:19.706048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.706048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:19.811143 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:19.811598 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:19.812445 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:19.812839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.813688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:19.813688 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.813688 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:19.813688 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.813688 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.813688 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:19.813688 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.813688 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:19.815124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:19.815969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:19.815969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.815969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:19.815969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.815969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.815969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:19.815969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.815969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:19.816962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.817455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:19.817957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.817957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:19.906362 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:19.906726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:19.907341 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:19.907602 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:19.907759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.908412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:19.908412 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.908412 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:19.908412 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.908412 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.908412 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:19.908412 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.908412 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.910043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:19.910678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:24.522561 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:24.523924 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:24.554676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:24.554937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.555301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:24.555301 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.555301 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:24.555301 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.555301 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.555301 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:24.555301 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.555301 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.556131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.557037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:24.674300 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:24.675106 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:24.675376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:24.675815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.676119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:24.676119 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.676119 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:24.676119 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.676119 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.676119 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:24.676119 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.676119 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.677392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.678314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:24.700164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:24.700455 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:24.700955 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:24.701245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.702040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:24.702040 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.702040 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:24.702040 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.702040 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.702040 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:24.702040 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.702040 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:24.703269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:24.704130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:24.704130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.704130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:24.704130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.704130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.704130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:24.704130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.704130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:24.705563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.706396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.707353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:24.707353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.707353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:24.811745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:24.812118 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:24.812685 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:24.813057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.813831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:24.813831 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.813831 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:24.813831 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.813831 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.813831 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:24.813831 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.813831 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:24.814713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:24.815486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:24.815486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.815486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:24.815486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.815486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.815486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:24.815486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.815486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:24.815831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.816521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:24.817379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.817379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:24.907174 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:24.908191 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:24.908776 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:24.909199 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:24.909479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.910300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:24.910300 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.910300 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:24.910300 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.910300 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.910300 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:24.910300 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.910300 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.911522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:24.912413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:29.523491 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:29.525333 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:29.560075 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:29.560544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.561450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:29.561450 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.561450 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:29.561450 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.561450 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.561450 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:29.561450 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.561450 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.562840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.563833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:29.674921 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:29.676307 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:29.676627 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:29.676902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.677769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:29.677769 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.677769 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:29.677769 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.677769 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.677769 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:29.677769 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.677769 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.679568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.680402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:29.700621 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:29.700874 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:29.701517 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:29.701803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.702520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:29.702520 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.702520 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:29.702520 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.702520 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.702520 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:29.702520 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.702520 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:29.703829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:29.704351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:29.704351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.704351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:29.704351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.704351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.704351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:29.704351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.704351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:29.705490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.705823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.706181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:29.706181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.706181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:29.812203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:29.812995 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:29.813551 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:29.813981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.817148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.817757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:29.818117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.818670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:29.819161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.819161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:29.907944 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:29.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:29.909383 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:29.909785 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:29.910035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.910823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:29.910823 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.910823 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:29.910823 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.910823 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.910823 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:29.910823 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.910823 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.912130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:29.913421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:34.524789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:34.526773 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:34.552497 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:31:34.557821 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:34.558093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.559409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:34.559409 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.559409 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:34.559409 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.559409 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.559409 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:34.559409 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.559409 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.560709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.561846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:34.675499 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:34.676793 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:34.677181 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:34.677523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.678239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:34.678239 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.678239 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:34.678239 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.678239 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.678239 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:34.678239 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.678239 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.679502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:34.701174 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:34.701565 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:34.702072 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:34.702787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.703176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:34.703176 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.703176 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:34.703176 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.703176 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.703176 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:34.703176 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.703176 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:34.704250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:34.704966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:34.704966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.704966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:34.704966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.704966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.704966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:34.704966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.704966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:34.705930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.706805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.707533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:34.707533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.707533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:34.812999 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:34.813436 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:34.814010 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:34.814316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.815066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:34.815066 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.815066 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:34.815066 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.815066 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.815066 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:34.815066 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.815066 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:34.816342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:34.817245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:34.817245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.817245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:34.817245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.817245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.817245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:34.817245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.817245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:34.818202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.819252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:34.819843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.819843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:34.908649 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:34.909119 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:34.909622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:34.910010 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:34.910245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.910983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:34.910983 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.910983 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:34.910983 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.910983 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.910983 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:34.910983 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.910983 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.912407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:34.912788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:39.525325 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:39.527559 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:39.562835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:39.563084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.563722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:39.563722 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.563722 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:39.563722 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.563722 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.563722 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:39.563722 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.563722 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.564589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.565072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:39.676242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:39.677468 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:39.678921 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:39.678921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.678921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:39.678921 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.678921 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:39.678921 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.678921 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.678921 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:39.678921 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.678921 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.680719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.681196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:39.701659 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:39.702773 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:39.702951 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:39.703165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.704315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:39.704315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.704315 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:39.704315 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.704315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.704315 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:39.704315 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.704315 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:39.705162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:39.705611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:39.705611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.705611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:39.705611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.705611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.705611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:39.705611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.705611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:39.706310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.706748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.707079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:39.707079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.707079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:39.813669 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:39.814389 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:39.815132 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:39.815753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.815954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:39.815954 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.815954 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:39.815954 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.815954 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.815954 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:39.815954 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.815954 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:39.817090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:39.818278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:39.818278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.818278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:39.818278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.818278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.818278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:39.818278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.818278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:39.818649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.819492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:39.820315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.820315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:39.909146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:39.909836 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:39.910240 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:39.910431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:39.910545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.911049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:39.911049 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.911049 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:39.911049 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.911049 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.911049 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:39.911049 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.911049 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.912131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:39.912903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:44.526547 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:44.527555 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:44.561031 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:44.561317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.561996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:44.561996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.561996 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:44.561996 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.561996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.561996 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:44.561996 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.561996 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.562898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.563365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:44.676976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:44.677876 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:44.678166 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:44.678391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.679112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:44.679112 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.679112 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:44.679112 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.679112 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.679112 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:44.679112 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.679112 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.683267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.683785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:44.702265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:44.703438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:44.703438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:44.703438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.704007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:44.704007 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.704007 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:44.704007 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.704007 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.704007 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:44.704007 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.704007 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:44.705358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:44.705772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:44.705772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.705772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:44.705772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.705772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.705772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:44.705772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.705772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:44.706823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.707175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.707609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:44.707609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.707609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:44.814607 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:44.814929 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:44.815453 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:44.815814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.816500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:44.816500 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.816500 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:44.816500 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.816500 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.816500 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:44.816500 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.816500 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:44.817521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:44.818170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:44.818170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.818170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:44.818170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.818170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.818170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:44.818170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.818170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:44.818417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.818950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:44.819546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.819546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:44.909936 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:44.910673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:44.911128 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:44.911414 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:44.911589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.912320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:44.912320 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.912320 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:44.912320 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.912320 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.912320 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:44.912320 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.912320 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.914077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:44.914621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:49.527092 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:49.535555 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:49.568356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:49.568615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.569462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:49.569462 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.569462 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:49.569462 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.569462 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.569462 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:49.569462 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.569462 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.570339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.570764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:49.677570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:49.678416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:49.678647 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:49.678823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.679508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:49.679508 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.679508 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:49.679508 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.679508 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.679508 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:49.679508 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.679508 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.680520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.680896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:49.702814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:49.703376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:49.703982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:49.704334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.705068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:49.705068 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.705068 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:49.705068 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.705068 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.705068 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:49.705068 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.705068 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:49.706407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:49.706770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:49.706770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.706770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:49.706770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.706770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.706770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:49.706770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.706770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:49.707957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.708325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.708658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:49.708658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.708658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:49.815306 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:49.815529 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:49.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:49.816415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.818637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:31:49.818637 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.818637 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:49.818637 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.818637 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.818637 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:49.818637 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.818637 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:49.821966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:49.822598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:49.823173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.823932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:49.910835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:49.911996 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:49.912755 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:49.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:49.914071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.915333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:49.915333 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.915333 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:49.915333 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.915333 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.915333 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:49.915333 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.915333 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.916432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:49.916949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:54.527955 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:54.529747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:54.556428 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:54.556582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.556987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:54.556987 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.556987 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:54.556987 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.556987 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.556987 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:54.556987 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.556987 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.557754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.558452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:54.678256 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:54.679829 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:54.680305 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:54.680630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.681893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:54.681893 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.681893 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:54.681893 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.681893 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.681893 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:54.681893 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.681893 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.683834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.684819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:54.703302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:54.703973 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:54.704364 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:54.704544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.704984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:54.704984 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.704984 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:54.704984 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.704984 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.704984 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:54.704984 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.704984 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:54.705763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.706545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:54.707305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:54.707959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:54.707959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:54.707959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:54.707959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:54.707959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.707959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:54.707959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.707959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.707959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:54.707959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.707959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:54.815818 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:54.816055 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:54.816472 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:54.816846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.817540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:31:54.817540 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.817540 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:54.817540 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.817540 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.817540 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:54.817540 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.817540 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:54.818118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:54.818924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:54.819746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.821020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:54.912172 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:54.912935 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:54.913368 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:54.913774 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:54.914051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.915154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:54.915154 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.915154 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:54.915154 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.915154 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.915154 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:54.915154 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.915154 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.917319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:54.918621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:59.528548 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:59.529421 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:59.564467 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:59.564968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.565038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:31:59.565038 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.565038 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:59.565038 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.565038 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.565038 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:59.565038 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.565038 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.565833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.566428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:59.679455 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:59.680472 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:59.680707 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:59.680846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.681414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:59.681414 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.681414 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:59.681414 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.681414 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.681414 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:59.681414 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.681414 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.682543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.682908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:59.704726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:59.705075 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:31:59.705845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:59.706252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.707109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:59.707109 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.707109 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:59.707109 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.707109 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.707109 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:59.707109 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.707109 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:59.707863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:59.708359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:59.708359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.708359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:59.708359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.708359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.708359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:59.708359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.708359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:31:59.708586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.709007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.709521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:59.709521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.709521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:59.816350 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:59.816883 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:59.817529 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:59.817813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.818423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:31:59.818423 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.818423 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:59.818423 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.818423 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.818423 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:59.818423 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.818423 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:31:59.820115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:31:59.821195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:59.821195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:31:59.821195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.821195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:31:59.821195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.821195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.821195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:31:59.821195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.821195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:31:59.822683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.823668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:59.824175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.824175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:31:59.912977 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:31:59.913687 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:31:59.914561 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:59.915121 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:31:59.915498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.916421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:31:59.916421 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.916421 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:31:59.916421 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.916421 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.916421 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:31:59.916421 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.916421 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.917524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:31:59.918532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:04.529405 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:04.530336 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:04.563921 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:04.564312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.565346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:04.565346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.565346 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:04.565346 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.565346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.565346 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:04.565346 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.565346 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.566475 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.567407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:04.680542 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:04.681041 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:04.681623 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:04.681834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.682598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:04.682598 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.682598 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:04.682598 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.682598 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.682598 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:04.682598 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.682598 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.683474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.684145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:04.705608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:04.705905 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:04.706604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:04.706942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.707829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:04.707829 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.707829 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:04.707829 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.707829 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.707829 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:04.707829 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.707829 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:04.708949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:04.709583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:04.709583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.709583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:04.709583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.709583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.709583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:04.709583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.709583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:04.710578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.711385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:04.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:04.816876 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:04.817502 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:04.818091 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:04.818361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.818988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:32:04.818988 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.818988 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:04.818988 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.818988 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.818988 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:04.818988 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.818988 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:04.819873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:04.820551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:04.820551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:04.820551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.820551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:04.820551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.820551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.820551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:04.820551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.820551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:04.821039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.821695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:04.822385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.822385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:04.914029 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:04.914479 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:04.915102 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:04.915495 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:04.915688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.916337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:04.916337 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.916337 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:04.916337 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.916337 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.916337 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:04.916337 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.916337 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.917503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:04.918163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:09.529862 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:09.531565 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:09.566861 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:09.567287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.568348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:09.568348 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.568348 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:09.568348 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.568348 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.568348 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:09.568348 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.568348 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.569524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.570880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:09.680838 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:09.682557 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:09.683425 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:09.683425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.684150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:09.684150 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.684150 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:09.684150 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.684150 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.684150 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:09.684150 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.684150 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.685353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.686252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:09.705912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:09.706556 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:09.707109 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:09.707993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.708239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:09.708239 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.708239 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:09.708239 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.708239 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.708239 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:09.708239 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.708239 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:09.709845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:09.710674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:09.710674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.710674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:09.710674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.710674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.710674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:09.710674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.710674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:09.711467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.712314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.713030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:09.713030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.713030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:09.817723 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:09.817861 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:09.818444 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:09.818709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.820317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.820805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:09.821269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.821717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:09.822201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.822201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:09.914625 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:09.915911 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:09.916696 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:09.917145 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:09.917399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.920561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:09.921130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:09.921700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:09.921700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:09.921700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:09.921700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:14.530576 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:14.531743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:14.574578 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:14.575002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.576005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:14.576005 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.576005 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:14.576005 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.576005 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.576005 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:14.576005 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.576005 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.577145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.578079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:14.681527 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:14.682524 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:14.683743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:14.684101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.685227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:14.685227 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.685227 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:14.685227 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.685227 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.685227 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:14.685227 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.685227 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.686825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.687314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:14.706863 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:14.707544 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:14.708438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:14.708759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.709649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:14.709649 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.709649 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:14.709649 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.709649 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.709649 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:14.709649 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.709649 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:14.711079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:14.711485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:14.711485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.711485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:14.711485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.711485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.711485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:14.711485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.711485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:14.712678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.713140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.713526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:14.713526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.713526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:14.818138 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:14.818633 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:14.818952 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:14.819205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.820344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:32:14.820344 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.820344 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:14.820344 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.820344 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.820344 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:14.820344 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.820344 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:14.821354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:14.822194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:14.822194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:14.822194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.822194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:14.822194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.822194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.822194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:14.822194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.822194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:14.822641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.823440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:14.824370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.824370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:14.915999 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:14.916694 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:14.917304 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:14.917570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:14.917707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.918566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:14.918566 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.918566 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:14.918566 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.918566 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.918566 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:14.918566 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.918566 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.920664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:14.921152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:19.531135 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:19.532283 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:19.566547 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:19.567996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.569008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:19.569008 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.569008 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:19.569008 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.569008 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.569008 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:19.569008 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.569008 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.570281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.571579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:19.682321 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:19.683349 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:19.683670 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:19.683903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.684865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:19.684865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.684865 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:19.684865 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.684865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.684865 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:19.684865 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.684865 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.686621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.687369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:19.707435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:19.707862 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:19.708761 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:19.709136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.710008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:19.710008 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.710008 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:19.710008 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.710008 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.710008 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:19.710008 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.710008 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:19.711307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:19.712090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:19.712090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.712090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:19.712090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.712090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.712090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:19.712090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.712090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:19.713001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.713770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.714316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:19.714316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.714316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:19.818847 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:19.819324 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:19.819920 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:19.820198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.820927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:32:19.820927 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.820927 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:19.820927 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.820927 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.820927 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:19.820927 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.820927 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:19.821850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:19.822317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:19.822317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:19.822317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.822317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:19.822317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.822317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.822317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:19.822317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.822317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:19.822930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.823308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:19.823660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.823660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:19.916775 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:19.917242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:19.917747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:19.917954 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:19.918066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.918661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:19.918661 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.918661 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:19.918661 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.918661 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.918661 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:19.918661 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.918661 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:19.921341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:24.531741 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:24.533398 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:24.566401 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:24.566753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.567490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:24.567490 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.567490 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:24.567490 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.567490 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.567490 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:24.567490 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.567490 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.568278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.568781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:24.682895 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:24.684230 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:24.684587 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:24.684834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.686480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:24.686480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.686480 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:24.686480 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.686480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.686480 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:24.686480 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.686480 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.688242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.689198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:24.707813 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:24.708016 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:24.708507 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:24.708835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.709943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:24.709943 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.709943 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:24.709943 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.709943 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.709943 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:24.709943 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.709943 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:24.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:24.711315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:24.711315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.711315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:24.711315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.711315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.711315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:24.711315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.711315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:24.711997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.712512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.713024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:24.713024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.713024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:24.819386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:24.820110 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:24.820303 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:24.820631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.821242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:32:24.821242 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.821242 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:24.821242 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.821242 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.821242 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:24.821242 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.821242 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:24.822070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:24.822572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:24.822572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:24.822572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.822572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:24.822572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.822572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.822572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:24.822572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.822572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:24.823103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.823568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:24.824079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.824079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:24.917519 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:24.917782 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:24.918265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:24.918636 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:24.918855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.919690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:24.919690 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.919690 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:24.919690 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.919690 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.919690 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:24.919690 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.919690 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.921186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:24.922194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:29.532375 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:29.533617 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:29.577826 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:29.578132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.579066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:29.579066 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.579066 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:29.579066 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.579066 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.579066 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:29.579066 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.579066 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.580327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.581151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:29.683703 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:29.684873 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:29.685203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:29.685479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.686406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:29.686406 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.686406 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:29.686406 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.686406 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.686406 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:29.686406 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.686406 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.687696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.688354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:29.708360 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:29.708862 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:29.709478 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:29.709769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.710547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:29.710547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.710547 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:29.710547 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.710547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.710547 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:29.710547 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.710547 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:29.711882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:29.712727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:29.712727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.712727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:29.712727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.712727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.712727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:29.712727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.712727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:29.713535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.714294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.715261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:29.715261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.715261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:29.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:29.820503 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:29.821242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:29.821751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.822753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:29.822753 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.822753 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:29.822753 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.822753 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.822753 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:29.822753 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.822753 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:29.823542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:29.823992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:29.823992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.823992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:29.823992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.823992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.823992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:29.823992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.823992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:29.824579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.825039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:29.825551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.825551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:29.918107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:29.918929 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:29.919920 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:29.920346 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:29.920666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.921952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:29.921952 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.921952 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:29.921952 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.921952 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.921952 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:29.921952 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.921952 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.923232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:29.924007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:34.533140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:34.534544 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:34.560999 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=236 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:32:34.575448 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:34.575709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.576553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:34.576553 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.576553 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:34.576553 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.576553 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.576553 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:34.576553 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.576553 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.577757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.579283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:34.684499 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:34.685770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:34.686297 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:34.686586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.687595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:34.687595 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.687595 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:34.687595 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.687595 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.687595 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:34.687595 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.687595 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.689315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.689789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:34.708859 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:34.709237 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:34.709646 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:34.709850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.710349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:34.710349 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.710349 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:34.710349 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.710349 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.710349 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:34.710349 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.710349 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:34.711202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:34.712040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:34.712040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.712040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:34.712040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.712040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.712040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:34.712040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.712040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:34.712487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.713207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.714354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:34.714354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.714354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:34.820945 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:34.821309 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:34.822040 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:34.822439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.823277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:34.823277 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.823277 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:34.823277 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.823277 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.823277 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:34.823277 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.823277 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:34.824573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:34.825205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:34.825205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.825205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:34.825205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.825205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.825205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:34.825205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.825205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:34.826429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.827100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:34.827794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.827794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:34.918925 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:34.919809 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:34.920514 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:34.920914 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:34.921138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.921927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:34.921927 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.921927 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:34.921927 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.921927 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.921927 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:34.921927 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.921927 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.923352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:34.924164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:39.356830 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:39.365142 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:389}}
Mar 23 22:32:39.365316 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=104232) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:370}}
Mar 23 22:32:39.365446 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Fixing resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:32:39.398498 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, resource={'id': '06cffee1-b8de-433e-a67a-ec0e950c9238', 'name': 'private', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['130adef4-306b-4397-a096-91ee5094d0e2', 'fb1912da-e258-4d41-8cd4-7896f0de6385'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:17:25Z', 'updated_at': '2026-03-23T22:17:28Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 36968}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.399003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.399721 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c4286b-d53b-4217-a9f3-05177afa9297, external_ids_update=None, columns={'tag': 36968}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.399721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.400551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:32:39.400551 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.400551 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:39.400551 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.400551 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.400551 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:39.400551 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.400551 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:39.401767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Failed to fix resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.401767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:32:39.401767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:32:39.401767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:32:39.401767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:32:39.401767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:32:39.401767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:32:39.401767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:39.401767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.401767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.401767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.402123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.402505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:32:39.402505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.402505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:32:39.402505 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Fixing resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:32:39.431887 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, resource={'id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'name': 'public', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'a53179c9-417a-4560-9bc7-e3f64551b15b'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-23T22:17:31Z', 'updated_at': '2026-03-23T22:17:40Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.432980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.432980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a8bd993e-d23e-4e73-8066-766307763bdb, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.433239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.434091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:32:39.434091 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.434091 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:39.434091 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.434091 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.434091 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:39.434091 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.434091 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:39.435542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Failed to fix resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.435542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:32:39.435542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:32:39.435542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:32:39.435542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:32:39.435542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:32:39.435542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:32:39.435542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:39.435542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.435542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.435542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.435899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.436370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:32:39.436370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.436370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:32:39.436370 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Fixing resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:32:39.463811 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, resource={'id': '3a042d54-7a79-465e-bd57-47cd6833a4d0', 'name': 'admin_net', 'tenant_id': 'b4ffb2af173e4612a30978ef510f4f44', 'project_id': 'b4ffb2af173e4612a30978ef510f4f44', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['935d20c1-7d6b-489a-8358-b5fb94945b7e'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:20:57Z', 'updated_at': '2026-03-23T22:20:58Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 47875}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.464064 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.464331 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c442d0-efff-4c7e-bcb6-377b4da39c34, external_ids_update=None, columns={'tag': 47875}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.464420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.465170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:32:39.465170 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.465170 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:39.465170 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.465170 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.465170 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:39.465170 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.465170 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:39.466116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Failed to fix resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.466116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:32:39.466116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:32:39.466116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:32:39.466116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:32:39.466116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:32:39.466116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:32:39.466116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:39.466116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.466116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.466116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.466532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.466889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:32:39.466889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.466889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:32:39.466889 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Fixing resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:32:39.497581 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, resource={'id': 'c9e6c30a-b30e-473b-b06a-50be75c90b36', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['421749b7-bb0d-4854-b3f4-f0414f4c37fb'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:21:34Z', 'updated_at': '2026-03-23T22:21:35Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 48210}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.497913 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.498297 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13535482-a705-4453-af87-e8a7d8c10c82, external_ids_update=None, columns={'tag': 48210}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.498551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.499136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:32:39.499136 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.499136 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:39.499136 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.499136 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.499136 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:39.499136 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.499136 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:39.500358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Failed to fix resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.500358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:32:39.500358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:32:39.500358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:32:39.500358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:32:39.500358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:32:39.500358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:32:39.500358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:39.500358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.500358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.500358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.500690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.501054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:32:39.501054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.501054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:32:39.501054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Fixing resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:32:39.534696 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:39.538749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:39.611539 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.611717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.612457 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, resource={'id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'enable_snat': True}, 'gw_port_id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-t62km', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'ip_address': '172.24.5.146', 'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a'}, {'ip_address': '2001:db8::396', 'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 3, 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d'}, resource_type=routers, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.612684 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.612905 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.613066 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.613177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.613805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:32:39.613805 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.613805 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:39.613805 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.613805 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.613805 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:39.613805 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.613805 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:39.613995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Unable to update router 154c08d6-78c5-49fd-a785-0df3f4277a23. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.615712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Failed to fix resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.615712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:32:39.615712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:32:39.615712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:32:39.615712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:32:39.615712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:32:39.615712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 23 22:32:39.615712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 23 22:32:39.615712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 23 22:32:39.615712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 23 22:32:39.615712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:32:39.616077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.616521 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:39.617089 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Fixing resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.618692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.619342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:39.685077 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:39.689181 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.689851 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.690080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.692002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:39.692002 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.692002 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:39.692002 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.692002 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.692002 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:39.692002 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.692002 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.693639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.694143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:39.709498 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:39.709973 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:39.712266 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.712964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.714188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:39.714188 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.714188 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:39.714188 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.714188 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.714188 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:39.714188 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.714188 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:39.715706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.717097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:39.717097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.717097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:39.717097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.717097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.717097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:39.717097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.717097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:39.719974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.720707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.721521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:39.721521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.721521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:39.749830 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, resource={'id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'name': '', 'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'project_id': '', 'mac_address': 'fa:16:3e:aa:0e:57', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160382', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.750416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a a53179c9-417a-4560-9bc7-e3f64551b15b', 'neutron:network_name': 'neutron-bb9d5c47-4694-42c2-a018-18e3c9058915', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-154c08d6-78c5-49fd-a785-0df3f4277a23'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::396/64', '172.24.5.146/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.750962 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=348aba7c-88a8-4a19-9486-df3f7edb4007, lrouter_port=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.751367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.752477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:32:39.752477 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.752477 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:39.752477 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.752477 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.752477 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:39.752477 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.752477 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:39.753990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Failed to fix resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.753990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:32:39.753990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:32:39.753990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:32:39.753990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:32:39.753990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:32:39.753990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 23 22:32:39.753990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:39.753990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.753990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.753990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.754538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.755092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:32:39.755092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.755092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:32:39.755092 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Maintenance task: Synchronization completed (took 0.39 seconds)
Mar 23 22:32:39.821488 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:39.821853 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:39.822633 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.823042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.823882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:39.823882 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.823882 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:39.823882 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.823882 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.823882 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:39.823882 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.823882 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:39.825228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.825651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:39.825651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.825651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:39.825651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.825651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.825651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:39.825651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.825651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:39.826658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.827117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:39.827611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.827611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:39.919900 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:39.920141 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:39.920778 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.921027 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:39.921187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.922011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:39.922011 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.922011 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:39.922011 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.922011 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.922011 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:39.922011 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.922011 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.923374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:39.924365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:44.536331 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:44.537799 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:44.571436 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:44.572348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.572677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:44.572677 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.572677 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:44.572677 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.572677 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.572677 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:44.572677 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.572677 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.573668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.574150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:44.685833 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:44.687333 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:44.687744 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:44.688008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.688981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:44.688981 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.688981 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:44.688981 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.688981 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.688981 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:44.688981 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.688981 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.690198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.690623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:44.709820 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:44.710063 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:44.710699 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:44.711001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.711787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:44.711787 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.711787 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:44.711787 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.711787 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.711787 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:44.711787 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.711787 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:44.712464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.713449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:44.714194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:44.715106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:44.715106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:44.715106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:44.715106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:44.715106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.715106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:44.715106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.715106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.715106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:44.715106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.715106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:44.822149 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:44.822688 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:44.823523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:44.824063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.825428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:44.825428 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.825428 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:44.825428 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.825428 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.825428 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:44.825428 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.825428 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:44.826623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:44.827060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:44.827060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.827060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:44.827060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.827060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.827060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:44.827060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.827060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:44.827740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.828244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:44.828741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.828741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:44.920228 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:44.920513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:44.921252 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:44.921678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:44.921908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.922927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:44.922927 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.922927 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:44.922927 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.922927 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.922927 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:44.922927 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.922927 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.924678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:44.925649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:49.537115 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:49.539453 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:49.575897 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:49.576361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.577257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:49.577257 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.577257 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:49.577257 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.577257 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.577257 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:49.577257 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.577257 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.578370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.578754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:49.686724 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:49.688406 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:49.688805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:49.689012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.690047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:49.690047 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.690047 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:49.690047 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.690047 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.690047 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:49.690047 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.690047 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.691884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.692669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:49.710470 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:49.711465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:49.712124 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:49.712579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.713552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:49.713552 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.713552 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:49.713552 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.713552 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.713552 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:49.713552 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.713552 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:49.714421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:49.715353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:49.715353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.715353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:49.715353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.715353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.715353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:49.715353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.715353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:49.716783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.717678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.718674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:49.718674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.718674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:49.822818 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:49.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:49.823447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:49.823775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.824442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:49.824442 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.824442 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:49.824442 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.824442 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.824442 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:49.824442 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.824442 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:49.825424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:49.825907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:49.825907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.825907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:49.825907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.825907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.825907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:49.825907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.825907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:49.826855 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.827497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:49.828039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.828039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:49.921451 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:49.921787 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:49.922333 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:49.922746 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:49.922746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.923731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:49.923731 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.923731 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:49.923731 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.923731 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.923731 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:49.923731 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.923731 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.925775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:49.926483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:54.537905 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:54.540020 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:54.568731 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:54.568959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.569744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:54.569744 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.569744 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:54.569744 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.569744 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.569744 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:54.569744 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.569744 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.571166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.572377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:54.687648 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:54.688827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:54.688967 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:54.689045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.689609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:54.689609 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.689609 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:54.689609 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.689609 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.689609 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:54.689609 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.689609 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.690657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.691317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:54.711134 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:54.711463 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:54.712559 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:54.712902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.713786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:54.713786 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.713786 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:54.713786 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.713786 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.713786 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:54.713786 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.713786 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:54.714827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:54.715328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:54.715328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.715328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:54.715328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.715328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.715328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:54.715328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.715328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:54.715889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.716595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.717153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:54.717153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.717153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:54.823692 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:54.824123 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:54.824806 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:54.825167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.825915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:54.825915 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.825915 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:54.825915 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.825915 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.825915 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:54.825915 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.825915 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:54.826805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:54.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:54.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:54.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:54.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:54.827643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.827973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:54.828338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.828338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:54.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:54.924271 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:54.925006 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:54.925269 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:54.925404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.926255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:54.926255 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.926255 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:54.926255 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.926255 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.926255 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:54.926255 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.926255 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.928424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:54.929366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:59.539066 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:59.540180 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:59.577192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:59.577806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.578354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:59.688680 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:59.690177 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:59.690508 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:59.690739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.691747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:59.691747 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.691747 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:59.691747 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.691747 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.691747 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:59.691747 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.691747 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.692631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.693027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:59.711923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:59.712542 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:32:59.713324 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:59.713656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.714304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:59.714304 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.714304 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:59.714304 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.714304 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.714304 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:59.714304 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.714304 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:59.715325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:59.715808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:59.715808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.715808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:59.715808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.715808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.715808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:59.715808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.715808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:32:59.716428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.716862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.717400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:59.717400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.717400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:59.824539 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:59.824883 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:59.825520 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:59.825766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.826346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:59.826346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.826346 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:59.826346 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.826346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.826346 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:59.826346 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.826346 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:32:59.827287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:59.828007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:32:59.828007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.828007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:32:59.828007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.828007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.828007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:32:59.828007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.828007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:32:59.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.829392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:59.830098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.830098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:32:59.924471 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:32:59.924776 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:32:59.925310 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:59.925516 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:32:59.925637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.926454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:32:59.926454 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.926454 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:32:59.926454 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.926454 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.926454 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:32:59.926454 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.926454 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.928163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:32:59.929114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:04.539818 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:04.540943 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:04.605798 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:04.606078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.606884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:04.606884 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.606884 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:04.606884 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.606884 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.606884 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:04.606884 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.606884 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.608068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.608746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:04.689389 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:04.690341 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:04.690541 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:04.690658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.691246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:04.691246 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.691246 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:04.691246 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.691246 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.691246 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:04.691246 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.691246 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.692285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.693062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:04.712676 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:04.717027 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:04.717836 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:04.718032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.718736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:04.718736 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.718736 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:04.718736 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.718736 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.718736 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:04.718736 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.718736 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:04.720319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:04.721175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:04.721175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.721175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:04.721175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.721175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.721175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:04.721175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.721175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:04.723203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.723974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.724813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:04.724813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.724813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:04.825100 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:04.825490 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:04.826064 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:04.826716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.827463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:04.827463 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.827463 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:04.827463 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.827463 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.827463 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:04.827463 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.827463 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:04.828431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:04.829341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:04.829341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.829341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:04.829341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.829341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.829341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:04.829341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.829341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:04.829706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.830579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:04.831528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.831528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:04.925112 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:04.925451 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:04.927005 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:04.927005 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:04.927005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.927155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:04.927155 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.927155 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:04.927155 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.927155 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.927155 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:04.927155 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.927155 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.928934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:04.929638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:09.540427 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:09.543259 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:09.581573 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:09.584250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.584250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:09.584250 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.584250 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:09.584250 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.584250 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.584250 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:09.584250 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.584250 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.584961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.585391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:09.690270 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:09.691235 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:09.691474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:09.691672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.692293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:09.692293 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.692293 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:09.692293 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.692293 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.692293 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:09.692293 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.692293 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.694470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.695340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:09.716669 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:09.717064 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:09.717604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:09.717874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.718546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:09.718546 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.718546 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:09.718546 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.718546 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.718546 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:09.718546 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.718546 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:09.719388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:09.719911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:09.719911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.719911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:09.719911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.719911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.719911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:09.719911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.719911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:09.720130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.720598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:09.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:09.825655 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:09.826071 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:09.826713 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:09.827022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.827693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Traceback (most recent call last):
Mar 23 22:33:09.827693 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.827693 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:09.827693 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.827693 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.827693 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:09.827693 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.827693 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:09.828563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:09.829013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:09.829013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:09.829013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.829013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:09.829013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.829013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.829013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:09.829013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.829013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:09.829627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.830109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:09.830626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.830626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:09.925710 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:09.925995 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:09.926588 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:09.926797 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:09.926916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.927469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:09.927469 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.927469 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:09.927469 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.927469 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.927469 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:09.927469 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.927469 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.929546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:09.930081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:14.541126 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:14.542289 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:14.577462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:14.577462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.577794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:14.577794 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.577794 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:14.577794 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.577794 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.577794 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:14.577794 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.577794 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.579113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.579857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:14.690752 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:14.692858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:14.692858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:14.692858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.693233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:14.693233 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.693233 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:14.693233 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.693233 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.693233 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:14.693233 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.693233 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.694957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.695345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:14.717170 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:14.717760 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:14.718357 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:14.718667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.719315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:14.719315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.719315 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:14.719315 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.719315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.719315 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:14.719315 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.719315 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:14.721075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.725040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:14.725402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:14.725766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:14.725766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:14.725766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:14.725766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:14.725766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.725766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:14.725766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.725766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.725766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:14.725766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.725766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:14.826270 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:14.826648 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:14.827225 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:14.827514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.828111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Traceback (most recent call last):
Mar 23 22:33:14.828111 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.828111 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:14.828111 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.828111 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.828111 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:14.828111 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.828111 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:14.829294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:14.829730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:14.829730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:14.829730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.829730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:14.829730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.829730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.829730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:14.829730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.829730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:14.830024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.830479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:14.830986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.830986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:14.926336 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:14.926803 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:14.927482 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:14.927744 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:14.927888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.928716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:14.928716 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.928716 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:14.928716 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.928716 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.928716 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:14.928716 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.928716 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.930477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:14.931261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:19.541718 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:19.543339 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:19.573202 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:19.573643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.574712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:19.574712 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.574712 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:19.574712 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.574712 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.574712 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:19.574712 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.574712 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.575878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.576443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:19.691412 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:19.693908 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:19.694151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:19.694345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.695073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:19.695073 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.695073 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:19.695073 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.695073 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.695073 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:19.695073 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.695073 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.696306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.696995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:19.717769 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:19.718125 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:19.718603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:19.718854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.719205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:19.719205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.719205 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:19.719205 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.719205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.719205 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:19.719205 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.719205 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:19.719986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:19.720829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:19.720829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.720829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:19.720829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.720829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.720829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:19.720829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.720829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:19.722115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.723566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.724082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:19.724082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.724082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:19.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:19.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:19.827767 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:19.828124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.828773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Traceback (most recent call last):
Mar 23 22:33:19.828773 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.828773 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:19.828773 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.828773 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.828773 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:19.828773 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.828773 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:19.829696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:19.830523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:19.830523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:19.830523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.830523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:19.830523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.830523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.830523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:19.830523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.830523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:19.830951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.831735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:19.832591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.832591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:19.926747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:19.927164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:19.927516 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:19.928780 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:19.928780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.928780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:19.928780 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.928780 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:19.928780 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.928780 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.928780 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:19.928780 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.928780 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.929082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:19.929582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:24.542340 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:24.543566 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:24.585709 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:24.586108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.586830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:24.586830 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.586830 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:24.586830 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.586830 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.586830 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:24.586830 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.586830 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.588009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.588997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:24.693517 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:24.694719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:24.695113 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:24.695571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.696067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:24.696067 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.696067 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:24.696067 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.696067 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.696067 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:24.696067 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.696067 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.697860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.698411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:24.718403 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:24.718703 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:24.719127 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:24.719384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.719928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:24.719928 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.719928 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:24.719928 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.719928 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.719928 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:24.719928 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.719928 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:24.720741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:24.721130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:24.721130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.721130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:24.721130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.721130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.721130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:24.721130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.721130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:24.721755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.722236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.722750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:24.722750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.722750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:24.827691 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:24.827971 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:24.828536 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:24.828724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.829209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Traceback (most recent call last):
Mar 23 22:33:24.829209 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.829209 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:24.829209 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.829209 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.829209 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:24.829209 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.829209 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:24.829978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:24.830729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:24.831530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.832364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:24.927648 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:24.928156 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:24.928820 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:24.929080 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:24.929233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.929796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:24.929796 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.929796 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:24.929796 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.929796 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.929796 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:24.929796 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.929796 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.930852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:24.931762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:29.542950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:29.544124 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:29.576646 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:29.577060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.578117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:29.578117 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.578117 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:29.578117 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.578117 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.578117 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:29.578117 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.578117 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.579308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.580203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:29.694153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:29.695158 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:29.695427 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:29.695764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.696455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:29.696455 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.696455 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:29.696455 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.696455 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.696455 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:29.696455 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.696455 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.697296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.697885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:29.718841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:29.719323 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:29.719927 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:29.720241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.720974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:29.720974 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.720974 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:29.720974 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.720974 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.720974 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:29.720974 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.720974 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:29.722755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:29.723533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:29.723533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.723533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:29.723533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.723533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.723533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:29.723533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.723533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:29.724319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.725012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.725878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:29.725878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.725878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:29.828322 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:29.828845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:29.829607 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:29.830209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.831435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Traceback (most recent call last):
Mar 23 22:33:29.831435 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.831435 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:29.831435 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.831435 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.831435 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:29.831435 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.831435 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:29.832625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:29.833408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:29.833408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:29.833408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.833408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:29.833408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.833408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.833408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:29.833408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.833408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-e45704ac-f9cc-460e-8258-eb8b793ea716 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:29.834324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.835154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:29.836153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.836153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:29.928477 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:29.929116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:29.929805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:29.930190 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:29.930479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.931430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:29.931430 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.931430 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:29.931430 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.931430 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.931430 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:29.931430 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.931430 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.933495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:29.934166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:34.543861 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:34.545409 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:34.568443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=432 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:33:34.583054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:34.583562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.584526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:34.584526 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.584526 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:34.584526 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.584526 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.584526 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:34.584526 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.584526 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.586239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.587290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:34.695260 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:34.696678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:34.697014 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:34.697333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.698182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:34.698182 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.698182 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:34.698182 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.698182 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.698182 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:34.698182 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.698182 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.700075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.700912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:34.719674 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:34.720471 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:34.721125 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:34.721690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.722358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:34.722358 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.722358 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:34.722358 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.722358 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.722358 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:34.722358 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.722358 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:34.723414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:34.724176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:34.724176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.724176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:34.724176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.724176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.724176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:34.724176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.724176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:34.724625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.725337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.726168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:34.726168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.726168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:34.829055 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:34.829408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:34.830171 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:34.830623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.831543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:34.831543 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.831543 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:34.831543 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.831543 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.831543 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:34.831543 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.831543 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:34.832443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:34.833136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:34.833136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.833136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:34.833136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.833136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.833136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:34.833136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.833136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:34.834043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.834794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:34.835845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.835845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:34.929319 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:34.929933 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:34.930768 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:34.931203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:34.931497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.932672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:34.932672 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.932672 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:34.932672 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.932672 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.932672 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:34.932672 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.932672 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.934808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:34.936030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:39.544645 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:39.546325 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:39.577132 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:39.577571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.578471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:39.578471 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.578471 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:39.578471 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.578471 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.578471 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:39.578471 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.578471 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.579623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.580827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:39.696249 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:39.697690 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:39.698023 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:39.698308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.699188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:39.699188 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.699188 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:39.699188 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.699188 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.699188 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:39.699188 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.699188 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.700932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.701834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:39.720199 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:39.720495 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:39.721103 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:39.721562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.722239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:39.722239 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.722239 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:39.722239 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.722239 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.722239 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:39.722239 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.722239 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:39.723128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:39.724051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:39.724051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.724051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:39.724051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.724051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.724051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:39.724051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.724051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:39.724477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.725241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.726136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:39.726136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.726136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:39.829742 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:39.830049 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:39.830837 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:39.831246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.832080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:39.832080 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.832080 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:39.832080 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.832080 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.832080 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:39.832080 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.832080 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:39.833435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:39.834198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:39.834198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.834198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:39.834198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.834198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.834198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:39.834198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.834198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:39.835003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.835858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:39.836763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.836763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:39.930353 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:39.930945 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:39.931533 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:39.931899 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:39.932124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.933122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:39.933122 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.933122 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:39.933122 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.933122 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.933122 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:39.933122 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.933122 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.934988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:39.935800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:44.545544 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:44.546672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:44.583252 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:44.583658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.584664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:44.584664 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.584664 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:44.584664 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.584664 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.584664 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:44.584664 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.584664 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.585890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.586822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:44.697583 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:44.699079 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:44.699453 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:44.699692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.700512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:44.700512 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.700512 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:44.700512 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.700512 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.700512 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:44.700512 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.700512 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.702164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.702929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:44.720542 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:44.720727 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:44.721243 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:44.721538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.722231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:44.722231 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.722231 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:44.722231 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.722231 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.722231 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:44.722231 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.722231 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:44.722985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:44.723736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:44.723736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.723736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:44.723736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.723736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.723736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:44.723736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.723736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:44.724076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.724789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.725607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:44.725607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.725607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:44.830463 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:44.831369 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:44.832368 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:44.832550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.833415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:44.833415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.833415 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:44.833415 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.833415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.833415 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:44.833415 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.833415 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:44.834461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:44.834966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:44.834966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.834966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:44.834966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.834966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.834966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:44.834966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.834966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:44.835495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.836126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:44.836660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.836660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:44.931590 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:44.932587 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:44.933323 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:44.933659 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:44.933943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.934843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:44.934843 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.934843 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:44.934843 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.934843 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.934843 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:44.934843 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.934843 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.936543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:44.937011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:49.546109 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:49.547403 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:49.580637 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:49.580944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.581814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:49.581814 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.581814 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:49.581814 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.581814 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.581814 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:49.581814 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.581814 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.583376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.584838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:49.698370 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:49.699800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:49.700188 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:49.700502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.701473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:49.701473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.701473 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:49.701473 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.701473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.701473 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:49.701473 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.701473 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.703266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.704098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:49.721023 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:49.721566 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:49.722108 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:49.722487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.723295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:49.723295 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.723295 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:49.723295 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.723295 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.723295 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:49.723295 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.723295 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:49.724814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:49.725614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:49.725614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.725614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:49.725614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.725614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.725614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:49.725614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.725614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:49.726614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.727499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.728351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:49.728351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.728351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:49.831604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:49.832149 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:49.832885 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:49.833353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.834169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:49.834169 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.834169 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:49.834169 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.834169 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.834169 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:49.834169 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.834169 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:49.835547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:49.836016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:49.836016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.836016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:49.836016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.836016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.836016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:49.836016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.836016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:49.837228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.837723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:49.838292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.838292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:49.932783 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:49.933313 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:49.934014 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:49.934427 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:49.934673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.935544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:49.935544 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.935544 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:49.935544 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.935544 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.935544 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:49.935544 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.935544 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.937895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:49.938732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:54.546980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:54.548646 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:54.585278 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:54.585278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.585827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:54.585827 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.585827 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:54.585827 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.585827 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.585827 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:54.585827 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.585827 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.587049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.587560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:54.699242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:54.700430 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:54.700748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:54.700994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.701878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:54.701878 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.701878 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:54.701878 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.701878 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.701878 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:54.701878 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.701878 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.704071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.705143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:54.721643 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:54.726178 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:54.727119 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:54.727516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.728333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:54.728333 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.728333 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:54.728333 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.728333 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.728333 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:54.728333 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.728333 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:54.729093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:54.729857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:54.729857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.729857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:54.729857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.729857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.729857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:54.729857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.729857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:54.730205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.730937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.731748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:54.731748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.731748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:54.832435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:54.832958 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:54.833598 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:54.834059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.834910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:54.834910 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.834910 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:54.834910 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.834910 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.834910 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:54.834910 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.834910 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:54.836063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:54.836867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:54.836867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.836867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:54.836867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.836867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.836867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:54.836867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.836867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:54.837701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.838598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:54.839526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.839526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:54.933521 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:54.934191 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:54.934866 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:54.935305 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:54.935544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.936369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:54.936369 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.936369 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:54.936369 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.936369 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.936369 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:54.936369 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.936369 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.937945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:54.938830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:59.547551 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:59.549195 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:59.584601 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:59.584924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.585549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:33:59.585549 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.585549 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:59.585549 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.585549 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.585549 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:59.585549 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.585549 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.586787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.587268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:59.699860 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:59.701191 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:59.701606 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:59.701868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.702755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:59.702755 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.702755 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:59.702755 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.702755 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.702755 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:59.702755 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.702755 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.704664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.705534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:59.725901 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:59.726494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:33:59.727054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:59.727388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.728079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:59.728079 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.728079 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:59.728079 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.728079 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.728079 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:59.728079 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.728079 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:59.729433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:59.730047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:59.730047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.730047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:59.730047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.730047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.730047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:59.730047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.730047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:33:59.730947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.731742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.732648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:59.732648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.732648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:59.833040 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:59.833416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:59.833843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:59.834119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.834750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:59.834750 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.834750 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:59.834750 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.834750 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.834750 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:59.834750 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.834750 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:33:59.836000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:59.836868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:33:59.836868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.836868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:33:59.836868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.836868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.836868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:33:59.836868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.836868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:33:59.837767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.838592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:59.839427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.839427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:33:59.934073 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:33:59.935029 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:33:59.935753 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:59.936291 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:33:59.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.937474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:33:59.937474 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.937474 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:33:59.937474 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.937474 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.937474 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:33:59.937474 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.937474 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.939497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:33:59.940402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:04.548586 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:04.550412 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:04.591121 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:04.591408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.591940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:04.591940 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.591940 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:04.591940 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.591940 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.591940 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:04.591940 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.591940 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.593126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.593967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:04.700870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:04.701843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:04.702158 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:04.702506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.703391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:04.703391 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.703391 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:04.703391 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.703391 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.703391 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:04.703391 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.703391 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.704613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.705477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:04.726564 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:04.727009 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:04.727545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:04.727833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.728591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:04.728591 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.728591 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:04.728591 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.728591 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.728591 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:04.728591 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.728591 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:04.729731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:04.730530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:04.730530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.730530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:04.730530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.730530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.730530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:04.730530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.730530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:04.731412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.732123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.732948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:04.732948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.732948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:04.833832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:04.834535 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:04.835165 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:04.835675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.836624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:04.836624 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.836624 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:04.836624 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.836624 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.836624 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:04.836624 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.836624 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:04.838168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:04.838936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:04.838936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.838936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:04.838936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.838936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.838936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:04.838936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.838936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:04.839829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.840586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:04.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:04.935439 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:04.936039 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:04.936656 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:04.937038 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:04.937369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.938185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:04.938185 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.938185 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:04.938185 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.938185 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.938185 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:04.938185 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.938185 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.940054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:04.940909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:09.549244 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:09.550192 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:09.592086 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:09.592432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.593129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:09.593129 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.593129 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:09.593129 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.593129 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.593129 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:09.593129 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.593129 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.594041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.594914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:09.702333 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:09.702873 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:09.703119 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:09.703453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.704354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:09.704354 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.704354 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:09.704354 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.704354 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.704354 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:09.704354 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.704354 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.706599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.707134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:09.727152 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:09.727695 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:09.727695 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:09.727893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.728349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:09.728349 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.728349 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:09.728349 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.728349 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.728349 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:09.728349 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.728349 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:09.729071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.730528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:09.731381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:09.732348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:09.732348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:09.732348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:09.732348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:09.732348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.732348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:09.732348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.732348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.732348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:09.732348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.732348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:09.834976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:09.835658 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:09.836423 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:09.836847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.837786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:09.837786 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.837786 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:09.837786 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.837786 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.837786 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:09.837786 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.837786 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:09.839389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:09.840198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:09.840198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.840198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:09.840198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.840198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.840198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:09.840198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.840198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:09.841188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.842086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:09.843004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.843004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:09.936162 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:09.936851 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:09.937512 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:09.937982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:09.938254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.939244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:09.939244 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.939244 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:09.939244 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.939244 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.939244 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:09.939244 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.939244 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.940979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:09.941841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:14.549827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:14.551143 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:14.591429 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:14.591705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.592358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:14.592358 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.592358 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:14.592358 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.592358 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.592358 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:14.592358 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.592358 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.593261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.594177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:14.702540 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:14.704176 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:14.704176 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:14.704491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.704977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:14.704977 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.704977 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:14.704977 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.704977 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.704977 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:14.704977 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.704977 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.706692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.707545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:14.727616 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:14.727982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:14.728788 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:14.729144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.730020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:14.730020 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.730020 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:14.730020 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.730020 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.730020 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:14.730020 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.730020 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:14.731150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:14.731705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:14.731705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.731705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:14.731705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.731705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.731705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:14.731705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.731705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:14.732761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.733781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.734761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:14.734761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.734761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:14.835813 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:14.836370 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:14.837160 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:14.837544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.838346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:14.838346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.838346 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:14.838346 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.838346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.838346 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:14.838346 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.838346 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:14.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:14.840464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:14.840464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.840464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:14.840464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.840464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.840464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:14.840464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.840464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:14.841666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.842503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:14.843384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.843384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:14.936955 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:14.937731 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:14.938462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:14.938839 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:14.939066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.939935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:14.939935 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.939935 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:14.939935 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.939935 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.939935 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:14.939935 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.939935 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.941806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:14.942877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:19.550522 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:19.552164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:19.582117 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:19.582531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.583396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:19.583396 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.583396 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:19.583396 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.583396 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.583396 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:19.583396 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.583396 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.584272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.585158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:19.703321 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:19.704727 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:19.705053 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:19.705338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.706181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:19.706181 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.706181 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:19.706181 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.706181 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.706181 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:19.706181 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.706181 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.707992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.708556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:19.728153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:19.728693 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:19.729252 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:19.729503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.730086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:19.730086 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.730086 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:19.730086 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.730086 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.730086 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:19.730086 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.730086 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:19.731339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:19.732092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:19.732092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.732092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:19.732092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.732092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.732092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:19.732092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.732092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:19.732919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.733780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.734592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:19.734592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.734592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:19.836610 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:19.836916 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:19.837658 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:19.837936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.838581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:34:19.838581 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.838581 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:19.838581 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.838581 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.838581 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:19.838581 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.838581 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:19.839713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:19.840428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:19.840428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:19.840428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.840428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:19.840428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.840428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.840428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:19.840428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.840428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:19.841446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.842203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:19.843039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.843039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:19.937646 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:19.938249 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:19.939081 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:19.939517 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:19.939738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.940735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:19.940735 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.940735 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:19.940735 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.940735 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.940735 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:19.940735 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.940735 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.942861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:19.943758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:24.551261 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:24.553260 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:24.585307 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:24.585574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.586375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:24.586375 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.586375 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:24.586375 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.586375 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.586375 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:24.586375 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.586375 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.587232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.587908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:24.706205 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:24.706927 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:24.707057 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:24.707254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.707627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:24.707627 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.707627 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:24.707627 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.707627 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.707627 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:24.707627 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.707627 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.708754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.709267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:24.728631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:24.729230 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:24.729829 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:24.730152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.730939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:24.730939 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.730939 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:24.730939 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.730939 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.730939 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:24.730939 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.730939 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:24.732462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:24.733304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:24.733304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.733304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:24.733304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.733304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.733304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:24.733304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.733304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:24.734414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.735102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.735578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:24.735578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.735578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:24.837192 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:24.837568 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:24.838086 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:24.838516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.839362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:34:24.839362 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.839362 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:24.839362 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.839362 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.839362 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:24.839362 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.839362 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:24.840576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:24.841273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:24.841273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:24.841273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.841273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:24.841273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.841273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.841273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:24.841273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.841273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:24.842234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.842981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:24.843513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.843513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:24.938127 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:24.938864 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:24.939402 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:24.939746 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:24.939984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.940763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:24.940763 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.940763 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:24.940763 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.940763 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.940763 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:24.940763 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.940763 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.942606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:24.943480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:29.552012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:29.554033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:29.594681 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:29.595106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.596009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:29.596009 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.596009 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:29.596009 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.596009 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.596009 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:29.596009 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.596009 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.597244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.598146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:29.706787 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:29.708165 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:29.708396 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:29.708646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.709588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:29.709588 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.709588 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:29.709588 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.709588 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.709588 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:29.709588 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.709588 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.711314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.712154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:29.729110 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:29.729644 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:29.730295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:29.730613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.731392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:29.731392 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.731392 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:29.731392 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.731392 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.731392 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:29.731392 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.731392 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:29.732702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:29.733624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:29.733624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.733624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:29.733624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.733624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.733624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:29.733624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.733624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:29.734478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.735139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.735914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:29.735914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.735914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:29.838147 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:29.838471 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:29.839049 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:29.839735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.841035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:34:29.841035 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.841035 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:29.841035 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.841035 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.841035 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:29.841035 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.841035 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:29.842344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:29.843144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:29.843144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:29.843144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.843144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:29.843144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.843144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.843144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:29.843144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.843144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:29.843661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.844433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:29.845273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.845273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:29.938734 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:29.939229 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:29.939796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:29.940363 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:29.940614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.941919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:29.941919 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.941919 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:29.941919 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.941919 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.941919 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:29.941919 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.941919 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.943375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:29.944144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:34.552770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:34.554451 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:34.574329 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:34:34.588690 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:34.589108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.590208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:34.590208 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.590208 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:34.590208 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.590208 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.590208 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:34.590208 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.590208 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.591308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.591976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:34.707801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:34.709664 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:34.709967 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:34.710516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.711369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:34.711369 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.711369 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:34.711369 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.711369 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.711369 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:34.711369 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.711369 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.712718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.713408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:34.729641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:34.730072 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:34.730606 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:34.731145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.732044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:34.732044 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.732044 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:34.732044 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.732044 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.732044 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:34.732044 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.732044 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:34.732747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:34.733434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:34.733434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.733434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:34.733434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.733434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.733434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:34.733434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.733434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:34.733684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.734286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.735027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:34.735027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.735027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:34.838737 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:34.839379 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:34.839979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:34.840648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.841368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:34:34.841368 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.841368 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:34.841368 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.841368 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.841368 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:34.841368 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.841368 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:34.842765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:34.843506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:34.843506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:34.843506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.843506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:34.843506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.843506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.843506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:34.843506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.843506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:34.844449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.845322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:34.846167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.846167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:34.939413 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:34.939852 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:34.940395 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:34.940788 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:34.941094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.942045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:34.942045 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.942045 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:34.942045 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.942045 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.942045 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:34.942045 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.942045 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.943646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:34.944500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:39.553452 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:39.555081 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:39.584844 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:39.585318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.586336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:39.586336 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.586336 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:39.586336 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.586336 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.586336 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:39.586336 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.586336 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.587315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.588172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:39.708922 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:39.710427 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:39.710694 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:39.710949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.712430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:39.712430 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.712430 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:39.712430 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.712430 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.712430 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:39.712430 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.712430 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.714131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.714909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:39.730423 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:39.730765 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:39.731491 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:39.731694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.732239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:39.732239 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.732239 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:39.732239 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.732239 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.732239 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:39.732239 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.732239 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:39.733476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:39.734084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:39.734084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.734084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:39.734084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.734084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.734084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:39.734084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.734084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:39.734398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.734858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.735393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:39.735393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.735393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:39.839635 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:39.840398 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:39.841345 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:39.841781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.842896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:34:39.842896 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.842896 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:39.842896 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.842896 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.842896 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:39.842896 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.842896 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:39.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:39.844803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:39.844803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:39.844803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.844803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:39.844803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.844803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.844803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:39.844803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.844803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:39.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.846891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:39.847732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.847732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:39.940128 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:39.943005 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:39.943906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:39.944385 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:39.945138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.945850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:39.945850 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.945850 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:39.945850 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.945850 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.945850 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:39.945850 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.945850 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.947577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:39.948114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:44.554111 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:44.555278 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:44.585231 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:44.585458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.586339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:44.586339 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.586339 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:44.586339 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.586339 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.586339 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:44.586339 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.586339 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.588695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.589557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:44.709885 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:44.711719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:44.712147 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:44.712448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.713601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:44.713601 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.713601 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:44.713601 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.713601 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.713601 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:44.713601 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.713601 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.715669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.716505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:44.730960 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:44.731434 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:44.732042 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:44.732428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.733171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:44.733171 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.733171 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:44.733171 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.733171 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.733171 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:44.733171 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.733171 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:44.734524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:44.735411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:44.735411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.735411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:44.735411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.735411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.735411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:44.735411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.735411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:44.736626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.737553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.738499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:44.738499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.738499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:44.840516 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:44.840990 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:44.841890 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:44.842364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.843451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:34:44.843451 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.843451 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:44.843451 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.843451 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.843451 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:44.843451 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.843451 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:44.844799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:44.845481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:44.845481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:44.845481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.845481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:44.845481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.845481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.845481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:44.845481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.845481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:44.846506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.847301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:44.848264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.848264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:44.942925 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:44.943749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:44.944180 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:44.944436 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:44.944596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.945379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:44.945379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.945379 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:44.945379 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.945379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.945379 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:44.945379 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.945379 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:44.948078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:49.554841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:49.556425 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:49.590314 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:49.590599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.591139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:49.591139 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.591139 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:49.591139 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.591139 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.591139 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:49.591139 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.591139 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.591992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.592888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:49.710484 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:49.712134 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:49.712521 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:49.713010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.713961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:49.713961 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.713961 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:49.713961 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.713961 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.713961 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:49.713961 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.713961 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.716566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.717740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:49.731337 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:49.732344 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:49.732344 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:49.732593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.733309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:49.733309 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.733309 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:49.733309 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.733309 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.733309 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:49.733309 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.733309 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:49.734884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:49.736100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:49.736100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.736100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:49.736100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.736100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.736100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:49.736100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.736100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:49.736780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.738052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.739168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:49.739168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.739168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:49.840984 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:49.841586 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:49.842083 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:49.842517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.843711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:34:49.843711 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.843711 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:49.843711 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.843711 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.843711 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:49.843711 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.843711 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:49.844758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:49.845485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:49.845485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:49.845485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.845485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:49.845485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.845485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.845485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:49.845485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.845485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:49.846337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.846828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:49.847347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.847347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:49.943565 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:49.944133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:49.944574 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:49.944828 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:49.944984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.945608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:49.945608 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.945608 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:49.945608 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.945608 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.945608 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:49.945608 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.945608 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.947353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:49.948166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:54.555507 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:54.557236 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:54.587302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:54.588029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.588444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:54.588444 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.588444 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:54.588444 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.588444 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.588444 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:54.588444 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.588444 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.589619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.590186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:54.711303 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:54.712164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:54.715626 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:54.715767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.716321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:54.716321 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.716321 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:54.716321 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.716321 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.716321 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:54.716321 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.716321 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.717313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.717762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:54.731797 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:54.732301 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:54.733123 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:54.733356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.734048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:54.734048 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.734048 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:54.734048 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.734048 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.734048 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:54.734048 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.734048 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:54.735466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:54.736677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:54.736677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.736677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:54.736677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.736677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.736677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:54.736677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.736677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:54.737069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.737819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.738863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:54.738863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.738863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:54.841543 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:54.842232 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:54.843049 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:54.843388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.844244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:34:54.844244 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.844244 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:54.844244 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.844244 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.844244 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:54.844244 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.844244 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:54.845771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:54.846549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:54.846549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:54.846549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.846549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:54.846549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.846549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.846549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:54.846549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.846549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:54.847715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.848736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:54.849606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.849606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:54.944143 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:54.944717 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:54.945391 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:54.945780 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:54.946018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.947170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:54.947170 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.947170 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:54.947170 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.947170 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.947170 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:54.947170 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.947170 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.948897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:54.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:59.556471 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:59.557502 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:59.591184 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:59.591511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.592139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:34:59.592139 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.592139 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:59.592139 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.592139 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.592139 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:59.592139 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.592139 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.593313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.594115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:59.712161 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:59.717824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:59.718022 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:59.718175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.718844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:59.718844 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.718844 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:59.718844 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.718844 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.718844 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:59.718844 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.718844 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.719962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.720555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:59.732266 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:59.732827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:34:59.733488 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:59.733806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.734593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:59.734593 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.734593 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:59.734593 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.734593 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.734593 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:59.734593 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.734593 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:59.736124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:59.737309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:59.737309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.737309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:59.737309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.737309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.737309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:59.737309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.737309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:34:59.737838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.738732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.739734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:59.739734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.739734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:59.842299 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:59.843237 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:59.844330 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:59.844790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:34:59.845869 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.845869 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:59.845869 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.845869 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.845869 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:59.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.845869 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:34:59.846734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:34:59.847792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:59.847792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:34:59.847792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.847792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:34:59.847792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.847792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.847792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:34:59.847792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.847792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:34:59.848463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.849727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:59.850373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.850373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:34:59.944933 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:34:59.945641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:34:59.945897 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:59.946296 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:34:59.946687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.947389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:34:59.947389 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.947389 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:34:59.947389 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.947389 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.947389 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:34:59.947389 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.947389 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.948952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:34:59.949469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:04.557270 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:04.558684 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:04.599233 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:04.599684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.600712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:04.600712 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.600712 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:04.600712 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.600712 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.600712 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:04.600712 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.600712 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.601922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.602734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:04.712662 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:04.713821 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:04.714086 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:04.714285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.714918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:04.714918 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.714918 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:04.714918 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.714918 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.714918 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:04.714918 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.714918 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.716016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.716801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:04.732780 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:04.733637 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:04.734056 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:04.734424 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.735357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:04.735357 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.735357 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:04.735357 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.735357 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.735357 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:04.735357 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.735357 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:04.736485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:04.737252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:04.737252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.737252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:04.737252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.737252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.737252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:04.737252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.737252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:04.738190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.738680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.739178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:04.739178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.739178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:04.842863 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:04.843479 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:04.844053 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:04.844541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.845573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:35:04.845573 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.845573 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:04.845573 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.845573 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.845573 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:04.845573 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.845573 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:04.847113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:04.847768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:04.847768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:04.847768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.847768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:04.847768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.847768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.847768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:04.847768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.847768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:04.849154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.849926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:04.850801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.850801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:04.945614 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:04.946015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:04.946488 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:04.946852 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:04.947078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.947835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:04.947835 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.947835 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:04.947835 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.947835 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.947835 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:04.947835 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.947835 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.949380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:04.950732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:09.558038 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:09.559682 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:09.592575 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:09.592921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.593840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:09.593840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.593840 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:09.593840 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.593840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.593840 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:09.593840 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.593840 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.594889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.595723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:09.713714 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:09.714926 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:09.715275 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:09.715592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.716676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:09.716676 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.716676 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:09.716676 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.716676 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.716676 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:09.716676 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.716676 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.717863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.718541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:09.733447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:09.733990 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:09.734614 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:09.734959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.735848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:09.735848 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.735848 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:09.735848 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.735848 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.735848 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:09.735848 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.735848 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:09.737301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:09.737882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:09.737882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.737882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:09.737882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.737882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.737882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:09.737882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.737882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:09.738944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.739690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.740529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:09.740529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.740529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:09.843484 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:09.844555 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:09.845022 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:09.845462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.846865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:35:09.846865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.846865 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:09.846865 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.846865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.846865 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:09.846865 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.846865 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:09.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:09.848540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:09.848540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:09.848540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.848540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:09.848540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.848540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.848540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:09.848540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.848540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:09.849558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:09.851294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.851294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:09.946173 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:09.946598 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:09.946930 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:09.947314 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:09.947543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.948249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:09.948249 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.948249 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:09.948249 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.948249 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.948249 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:09.948249 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.948249 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.949723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:09.950560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:14.558796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:14.560153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:14.594941 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:14.595420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.596409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:14.596409 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.596409 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:14.596409 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.596409 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.596409 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:14.596409 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.596409 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.597665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.598575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:14.714456 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:14.715679 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:14.715862 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:14.716098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.716895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:14.716895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.716895 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:14.716895 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.716895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.716895 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:14.716895 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.716895 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.718653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.719137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:14.733915 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:14.734505 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:14.735569 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:14.735569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.736577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:14.736577 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.736577 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:14.736577 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.736577 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.736577 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:14.736577 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.736577 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:14.737936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:14.738825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:14.738825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.738825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:14.738825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.738825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.738825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:14.738825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.738825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:14.739489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.740381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.741289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:14.741289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.741289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:14.844208 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:14.844753 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:14.845490 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:14.845868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.846819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:35:14.846819 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.846819 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:14.846819 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.846819 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.846819 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:14.846819 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.846819 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:14.848007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:14.848791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:14.848791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:14.848791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.848791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:14.848791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.848791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.848791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:14.848791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.848791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:14.849644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.850470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:14.851358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.851358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:14.946880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:14.947319 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:14.947769 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:14.948145 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:14.948405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.949093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:14.949093 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.949093 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:14.949093 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.949093 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.949093 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:14.949093 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.949093 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.950585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:14.951489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:19.559615 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:19.561326 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:19.598668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:19.599011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.599772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:19.599772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.599772 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:19.599772 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.599772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.599772 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:19.599772 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.599772 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.600525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.601099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:19.715126 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:19.716263 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:19.716598 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:19.716833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.717737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:19.717737 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.717737 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:19.717737 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.717737 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.717737 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:19.717737 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.717737 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.719430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.720264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:19.734571 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:19.735067 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:19.735633 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:19.735936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.737486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:19.737486 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.737486 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:19.737486 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.737486 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.737486 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:19.737486 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.737486 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:19.738883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:19.739887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:19.739887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.739887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:19.739887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.739887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.739887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:19.739887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.739887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:19.740631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.741184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.741935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:19.741935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.741935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:19.844907 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:19.845402 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:19.846157 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:19.846605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.847461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:35:19.847461 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.847461 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:19.847461 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.847461 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.847461 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:19.847461 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.847461 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:19.848829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:19.849609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:19.849609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:19.849609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.849609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:19.849609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.849609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.849609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:19.849609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.849609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:19.850697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.851553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:19.852399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.852399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:19.947356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:19.947599 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:19.948153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:19.948537 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:19.948737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.949542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:19.949542 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.949542 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:19.949542 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.949542 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.949542 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:19.949542 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.949542 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.951094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:19.952282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:24.560367 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:24.561628 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:24.593958 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:24.594173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.594730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:24.594730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.594730 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:24.594730 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.594730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.594730 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:24.594730 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.594730 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.595625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.596182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:24.715819 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:24.717169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:24.717535 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:24.717766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.718652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:24.718652 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.718652 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:24.718652 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.718652 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.718652 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:24.718652 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.718652 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.720033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.720700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:24.735015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:24.735568 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:24.736270 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:24.736571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.737247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:24.737247 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.737247 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:24.737247 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.737247 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.737247 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:24.737247 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.737247 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:24.738711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.739782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:24.740693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:24.741714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:24.741714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:24.741714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:24.741714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:24.741714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.741714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:24.741714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.741714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.741714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:24.741714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.741714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:24.845518 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:24.845826 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:24.846725 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:24.847051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.847967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:35:24.847967 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.847967 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:24.847967 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.847967 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.847967 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:24.847967 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.847967 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:24.848877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:24.849631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:24.849631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:24.849631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.849631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:24.849631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.849631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.849631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:24.849631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.849631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:24.850028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.850827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:24.851671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.851671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:24.947950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:24.948313 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:24.949037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:24.949590 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:24.949848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.951288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:24.951288 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.951288 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:24.951288 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.951288 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.951288 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:24.951288 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.951288 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.953343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:24.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:29.561002 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:29.562683 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:29.591887 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:29.592356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.592725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:29.592725 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.592725 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:29.592725 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.592725 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.592725 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:29.592725 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.592725 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.593789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.594776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:29.716519 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:29.717557 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:29.717951 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:29.718328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.719349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:29.719349 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.719349 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:29.719349 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.719349 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.719349 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:29.719349 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.719349 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.720517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.721343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:29.735615 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:29.736233 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:29.736660 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:29.736981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.737830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:29.737830 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.737830 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:29.737830 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.737830 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.737830 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:29.737830 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.737830 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:29.739624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.740758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:29.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:29.743119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:29.743119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:29.743119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:29.743119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:29.743119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.743119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:29.743119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.743119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.743119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:29.743119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.743119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:29.845982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:29.846545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:29.847202 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:29.847714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.848702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:35:29.848702 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.848702 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:29.848702 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.848702 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.848702 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:29.848702 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.848702 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:29.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:29.850795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:29.850795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:29.850795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.850795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:29.850795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.850795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.850795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:29.850795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.850795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:29.851817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.852612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:29.853389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.853389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:29.948579 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:29.949107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:29.949815 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:29.950195 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:29.950535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.951454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:29.951454 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.951454 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:29.951454 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.951454 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.951454 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:29.951454 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.951454 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.953252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:29.954026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:34.561766 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:34.563465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:34.580884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=235 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:35:34.599544 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:34.599824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.600473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:34.600473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.600473 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:34.600473 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.600473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.600473 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:34.600473 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.600473 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.601435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.602492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:34.717282 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:34.718819 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:34.719164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:34.719440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.720482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:34.720482 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.720482 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:34.720482 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.720482 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.720482 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:34.720482 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.720482 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.722297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.722865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:34.736697 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:34.737163 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:34.737729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:34.738018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.738730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:34.738730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.738730 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:34.738730 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.738730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.738730 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:34.738730 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.738730 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:34.740471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.741810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:34.742794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:34.743661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:34.743661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:34.743661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:34.743661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:34.743661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.743661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:34.743661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.743661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.743661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:34.743661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.743661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:34.846612 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:34.847004 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:34.847604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:34.847891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.848583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:35:34.848583 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.848583 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:34.848583 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.848583 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.848583 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:34.848583 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.848583 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:34.850081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:34.850797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:34.850797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:34.850797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.850797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:34.850797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.850797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.850797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:34.850797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.850797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:34.852248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.853075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:34.853708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.853708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:34.949184 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:34.949673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:34.950358 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:34.950756 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:34.950985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.951807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:34.951807 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.951807 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:34.951807 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.951807 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.951807 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:34.951807 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.951807 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.953949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:34.954727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:39.562434 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:39.564013 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:39.599876 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:39.600134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.600867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:39.600867 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.600867 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:39.600867 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.600867 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.600867 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:39.600867 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.600867 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.602038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.602894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:39.717988 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:39.719129 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:39.719555 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:39.719829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.720744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:39.720744 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.720744 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:39.720744 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.720744 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.720744 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:39.720744 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.720744 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.722507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.723702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:39.737144 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:39.737654 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:39.738044 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:39.738232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.738758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:39.738758 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.738758 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:39.738758 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.738758 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.738758 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:39.738758 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.738758 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:39.740431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:39.741517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:39.741517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.741517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:39.741517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.741517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.741517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:39.741517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.741517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:39.742169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.743105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.744316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:39.744316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.744316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:39.847305 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:39.847979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:39.848608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:39.849014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:35:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:39.851373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:39.852048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:39.852048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:39.852048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.852048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:39.852048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.852048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.852048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:39.852048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.852048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:39.853003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.853766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:39.854600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.854600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:39.949719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:39.950324 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:39.950746 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:39.950995 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:39.951180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.952028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:39.952028 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.952028 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:39.952028 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.952028 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.952028 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:39.952028 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.952028 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.953190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:39.954039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:44.563263 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:44.564513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:44.595704 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:44.595925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.596695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:44.596695 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.596695 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:44.596695 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.596695 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.596695 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:44.596695 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.596695 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.597625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.598583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:44.718619 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:44.719800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:44.720031 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:44.720189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.720853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:44.720853 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.720853 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:44.720853 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.720853 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.720853 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:44.720853 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.720853 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.722261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.722998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:44.737667 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:44.738054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:44.738845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:44.738845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.739375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:44.739375 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.739375 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:44.739375 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.739375 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.739375 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:44.739375 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.739375 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:44.740561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:44.741066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:44.741066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.741066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:44.741066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.741066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.741066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:44.741066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.741066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:44.742052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.742695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.743173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:44.743173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.743173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:44.848015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:44.848673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:44.849347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:44.849738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.850813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:35:44.850813 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.850813 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:44.850813 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.850813 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.850813 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:44.850813 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.850813 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:44.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:44.853492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:44.853492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:44.853492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.853492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:44.853492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.853492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.853492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:44.853492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.853492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:44.854705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.855159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:44.855658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.855658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:44.950372 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:44.950812 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:44.951606 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:44.951776 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:44.951979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.952799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:44.952799 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.952799 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:44.952799 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.952799 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.952799 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:44.952799 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.952799 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.954304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:44.955285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:49.564097 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:49.566103 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:49.599725 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:49.600801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.601800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:49.601800 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.601800 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:49.601800 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.601800 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.601800 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:49.601800 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.601800 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.602701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.603554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:49.719506 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:49.721003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:49.721407 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:49.721805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.722605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:49.722605 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.722605 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:49.722605 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.722605 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.722605 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:49.722605 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.722605 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.724409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.725525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:49.738245 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:49.738751 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:49.739462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:49.739795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.740681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:49.740681 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.740681 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:49.740681 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.740681 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.740681 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:49.740681 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.740681 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:49.741917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:49.742785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:49.742785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.742785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:49.742785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.742785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.742785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:49.742785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.742785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:49.743430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.744181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.745091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:49.745091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.745091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:49.848631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:49.849169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:49.849803 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:49.850173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.851020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:35:49.851020 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.851020 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:49.851020 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.851020 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.851020 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:49.851020 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.851020 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:49.852250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:49.853007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:49.853007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:49.853007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.853007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:49.853007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.853007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.853007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:49.853007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.853007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:49.853886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.854715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:49.855574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.855574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:49.950843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:49.951109 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:49.951630 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:49.952011 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:49.952368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.953198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:49.953198 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.953198 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:49.953198 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.953198 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.953198 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:49.953198 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.953198 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.954154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:49.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:54.564985 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:54.566150 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:54.602708 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:54.603422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.604253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:54.604253 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.604253 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:54.604253 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.604253 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.604253 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:54.604253 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.604253 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.605611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.606552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:54.720475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:54.722142 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:54.722596 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:54.723358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.725504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:54.725504 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.725504 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:54.725504 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.725504 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.725504 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:54.725504 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.725504 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.727599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.728323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:54.738741 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:54.739250 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:54.739892 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:54.740124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.741329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:54.741329 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.741329 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:54.741329 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.741329 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.741329 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:54.741329 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.741329 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:54.742248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:54.743192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:54.743192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.743192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:54.743192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.743192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.743192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:54.743192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.743192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:54.744844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.745589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.746452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:54.746452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.746452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:54.849183 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:54.849530 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:54.850021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:54.850337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.850901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:54.850901 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.850901 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:54.850901 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.850901 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.850901 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:54.850901 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.850901 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:54.851819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:54.852582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:54.852582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.852582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:54.852582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.852582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.852582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:54.852582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.852582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:54.852928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.853690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:54.854535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.854535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:54.951496 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:54.952173 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:54.952909 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:54.952909 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:54.953382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.954409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:54.954409 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.954409 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:54.954409 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.954409 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.954409 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:54.954409 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.954409 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.955568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:54.956249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:59.565650 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:59.566814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:59.602774 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:59.602774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:59.603984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:59.604491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.604848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:59.604848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.604848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.604848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:59.604848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.604848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:59.721163 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:59.722489 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:59.723445 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:59.723445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.723445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:59.723445 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.723445 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:59.723445 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.723445 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.723445 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:59.723445 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.723445 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.724312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.724720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:59.739340 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:59.739726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:35:59.741024 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:59.741024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.741123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:59.741123 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.741123 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:59.741123 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.741123 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.741123 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:59.741123 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.741123 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:59.741803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:59.742299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:59.742299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.742299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:59.742299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.742299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.742299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:59.742299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.742299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:35:59.742511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.743164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:59.743164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.743164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:59.849609 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:59.850130 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.852735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.853180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:59.853710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.854347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.854347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:59.854347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.854347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:35:59.952193 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:35:59.952767 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:35:59.954767 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:59.954767 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:35:59.954767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.954979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:35:59.954979 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.954979 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:35:59.954979 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.954979 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.954979 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:35:59.954979 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.954979 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.956178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:35:59.956982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:04.566386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:04.568352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:04.601432 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:04.601432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.602240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:04.602240 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.602240 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:04.602240 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.602240 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.602240 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:04.602240 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.602240 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.602872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.603371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:04.721814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:04.723253 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:04.723628 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:04.724052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.724846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:04.724846 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.724846 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:04.724846 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.724846 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.724846 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:04.724846 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.724846 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.726664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.727155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:04.739772 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:04.740253 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:04.740976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:04.741398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.742364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:04.742364 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.742364 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:04.742364 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.742364 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.742364 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:04.742364 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.742364 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:04.743521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:04.743935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:04.743935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.743935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:04.743935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.743935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.743935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:04.743935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.743935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:04.745016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.745485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.746004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:04.746004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.746004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:04.850110 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:04.850616 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:04.851338 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:04.851763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.852639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:04.852639 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.852639 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:04.852639 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.852639 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.852639 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:04.852639 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.852639 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:04.853943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:04.854461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:04.854461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.854461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:04.854461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.854461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.854461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:04.854461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.854461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:04.855622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.856133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:04.856769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.856769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:04.952977 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:04.953452 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:04.953917 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:04.954311 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:04.954548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.955228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:04.955228 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.955228 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:04.955228 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.955228 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.955228 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:04.955228 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.955228 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.956144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:04.957023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:09.567315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:09.569311 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:09.604352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:09.604535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.605178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:09.605178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.605178 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:09.605178 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.605178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.605178 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:09.605178 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.605178 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.606265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.607063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:09.722533 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:09.723662 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:09.723901 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:09.725954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:09.726674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:09.727657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.727657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:09.727657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.727657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.727657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:09.727657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.727657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:09.740275 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:09.740759 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:09.741895 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:09.742235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.743249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:09.743249 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.743249 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:09.743249 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.743249 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.743249 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:09.743249 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.743249 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:09.744437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:09.744910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:09.744910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.744910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:09.744910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.744910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.744910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:09.744910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.744910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:09.745967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.746415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.746924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:09.746924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.746924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:09.851095 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:09.851312 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:09.851941 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:09.852234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.852877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:09.852877 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.852877 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:09.852877 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.852877 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.852877 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:09.852877 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.852877 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:09.854193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:09.854692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:09.854692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.854692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:09.854692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.854692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.854692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:09.854692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.854692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:09.856187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.856653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:09.857179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.857179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:09.953748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:09.954202 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:09.954511 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:09.954776 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:09.955013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.955605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:09.955605 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.955605 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:09.955605 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.955605 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.955605 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:09.955605 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.955605 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.956436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:09.956928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:14.568034 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:14.569793 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:14.605470 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:14.605691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.606131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:14.606131 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.606131 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:14.606131 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.606131 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.606131 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:14.606131 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.606131 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.607102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.607876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:14.723429 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:14.724705 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:14.725138 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:14.725421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.726296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:14.726296 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.726296 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:14.726296 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.726296 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.726296 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:14.726296 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.726296 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.728050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.728933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:14.741192 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:14.741641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:14.742203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:14.742556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.743235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:14.743235 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.743235 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:14.743235 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.743235 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.743235 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:14.743235 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.743235 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:14.744090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:14.744875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:14.744875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.744875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:14.744875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.744875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.744875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:14.744875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.744875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:14.745318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.746058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.746885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:14.746885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.746885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:14.851636 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:14.851858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:14.852481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:14.852863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.853660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:14.853660 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.853660 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:14.853660 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.853660 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.853660 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:14.853660 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.853660 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:14.854933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:14.855721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:14.855721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.855721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:14.855721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.855721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.855721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:14.855721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.855721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:14.856527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.857367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:14.858354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.858354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:14.954353 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:14.954743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:14.955313 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:14.955662 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:14.955883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.956597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:14.956597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.956597 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:14.956597 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.956597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.956597 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:14.956597 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.956597 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.958263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:14.959031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:19.568738 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:19.570652 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:19.606579 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:19.606796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.607162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:19.607162 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.607162 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:19.607162 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.607162 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.607162 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:19.607162 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.607162 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.607939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.608701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:19.724111 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:19.725533 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:19.725869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:19.726102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.726981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:19.726981 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.726981 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:19.726981 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.726981 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.726981 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:19.726981 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.726981 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.728703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.730281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:19.742345 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:19.742920 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:19.743471 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:19.743672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.744242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:19.744242 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.744242 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:19.744242 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.744242 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.744242 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:19.744242 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.744242 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:19.745544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.747087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:19.748155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:19.749368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:19.749368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:19.749368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:19.749368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:19.749368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.749368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:19.749368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.749368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.749368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:19.749368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.749368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:19.852167 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:19.852673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:19.853110 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:19.853452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.854016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:19.854016 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.854016 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:19.854016 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.854016 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.854016 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:19.854016 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.854016 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:19.854746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:19.855566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:19.855566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.855566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:19.855566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.855566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.855566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:19.855566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.855566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:19.855876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.856754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:19.858621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.858621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:19.954843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:19.955236 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:19.955924 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:19.956351 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:19.956571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.957356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:19.957356 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.957356 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:19.957356 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.957356 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.957356 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:19.957356 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.957356 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.958858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:19.960325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:24.569504 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:24.571018 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:24.606978 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:24.607397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.608243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:24.608243 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.608243 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:24.608243 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.608243 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.608243 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:24.608243 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.608243 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.609414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.610822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:24.724952 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:24.729262 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:24.729540 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:24.729864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.730560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:24.730560 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.730560 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:24.730560 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.730560 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.730560 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:24.730560 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.730560 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.732156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.733003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:24.743010 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:24.743333 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:24.743803 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:24.744127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.744640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:24.744640 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.744640 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:24.744640 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.744640 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.744640 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:24.744640 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.744640 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:24.745500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:24.746149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:24.746149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.746149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:24.746149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.746149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.746149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:24.746149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.746149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:24.746559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.747107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.747814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:24.747814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.747814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:24.852943 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:24.853759 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:24.854334 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:24.854673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.855343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:24.855343 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.855343 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:24.855343 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.855343 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.855343 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:24.855343 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.855343 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:24.856631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:24.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:24.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:24.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:24.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:24.857987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.858481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:24.858998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.858998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:24.955489 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:24.956093 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:24.956634 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:24.957008 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:24.957255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.958001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:24.958001 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.958001 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:24.958001 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.958001 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.958001 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:24.958001 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.958001 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.959709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:24.960531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:29.570503 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:29.572164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:29.611453 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:29.612459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.613128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:29.613128 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.613128 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:29.613128 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.613128 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.613128 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:29.613128 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.613128 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.613767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.614693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:29.728928 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:29.730138 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:29.730805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:29.730805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.731390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:29.731390 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.731390 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:29.731390 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.731390 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.731390 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:29.731390 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.731390 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.732495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.733320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:29.743442 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:29.743836 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:29.744246 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:29.744470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.745104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:29.745104 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.745104 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:29.745104 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.745104 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.745104 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:29.745104 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.745104 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:29.746209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:29.746744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:29.746744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.746744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:29.746744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.746744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.746744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:29.746744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.746744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:29.747852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.748363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.748841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:29.748841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.748841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:29.853917 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:29.854171 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:29.854957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:29.855390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.855711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:29.855711 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.855711 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:29.855711 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.855711 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.855711 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:29.855711 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.855711 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:29.857093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:29.857555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:29.857555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.857555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:29.857555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.857555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.857555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:29.857555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.857555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:29.858666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.859470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:29.860279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.860279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:29.956151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:29.956582 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:29.957416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:29.957866 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:29.958115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.959106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:29.959106 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.959106 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:29.959106 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.959106 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.959106 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:29.959106 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.959106 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.960865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:29.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:34.571341 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:34.573013 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:34.588002 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=239 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:36:34.605786 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:34.606077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.606700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:34.606700 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.606700 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:34.606700 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.606700 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.606700 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:34.606700 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.606700 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.607586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.608474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:34.729810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:34.730972 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:34.731304 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:34.731586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.732480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:34.732480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.732480 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:34.732480 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.732480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.732480 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:34.732480 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.732480 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.734046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.734846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:34.743947 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:34.744247 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:34.744685 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:34.744892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.745421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:34.745421 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.745421 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:34.745421 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.745421 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.745421 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:34.745421 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.745421 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:34.746683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:34.747457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:34.747457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.747457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:34.747457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.747457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.747457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:34.747457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.747457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:34.748145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.749074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.749931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:34.749931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.749931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:34.854564 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:34.854911 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:34.855672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:34.856183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.856986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:34.856986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.856986 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:34.856986 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.856986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.856986 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:34.856986 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.856986 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:34.857869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:34.858594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:34.858594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.858594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:34.858594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.858594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.858594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:34.858594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.858594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:34.858910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.859606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:34.860389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.860389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:34.956882 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:34.957302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:34.957811 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:34.958319 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:34.958549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.959129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:34.959129 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.959129 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:34.959129 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.959129 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.959129 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:34.959129 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.959129 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.960157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:34.961034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:39.571921 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:39.573011 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:39.615053 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:39.615053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.615053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:39.615053 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.615053 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:39.615053 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.615053 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.615053 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:39.615053 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.615053 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.616027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.616684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:39.730474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:39.732379 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:39.733076 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:39.733076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.733387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:39.733387 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.733387 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:39.733387 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.733387 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.733387 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:39.733387 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.733387 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.734428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.734790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:39.744519 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:39.744933 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:39.745423 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:39.745606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.746197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:39.746197 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.746197 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:39.746197 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.746197 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.746197 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:39.746197 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.746197 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:39.747629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:39.748018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:39.748018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.748018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:39.748018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.748018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.748018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:39.748018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.748018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:39.749112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.749481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.749844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:39.749844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.749844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:39.855265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:39.855555 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:39.856104 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:39.856402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.856944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:39.856944 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.856944 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:39.856944 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.856944 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.856944 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:39.856944 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.856944 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:39.857634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.858524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:39.859050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:39.859670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:39.859670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:39.859670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.859670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:39.859670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.859670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.859670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:39.859670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.859670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:39.957552 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:39.958339 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:39.958860 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:39.959069 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:39.959177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.959634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:39.959634 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.959634 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:39.959634 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.959634 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.959634 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:39.959634 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.959634 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.960453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:39.960842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:44.572687 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:44.574531 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:44.611182 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:44.611624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.612450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:44.612450 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.612450 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:44.612450 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.612450 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.612450 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:44.612450 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.612450 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.613414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.614326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:44.731196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:44.732671 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:44.732959 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:44.733517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.734063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:44.734063 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.734063 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:44.734063 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.734063 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.734063 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:44.734063 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.734063 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.735955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.737184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:44.744955 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:44.745170 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:44.745786 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:44.746078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.746749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:44.746749 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.746749 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:44.746749 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.746749 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.746749 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:44.746749 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.746749 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:44.747626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:44.748487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:44.748487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.748487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:44.748487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.748487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.748487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:44.748487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.748487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:44.748892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.749650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.750460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:44.750460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.750460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:44.855947 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:44.856436 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:44.857312 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:44.857554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.858315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:44.858315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.858315 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:44.858315 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.858315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.858315 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:44.858315 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.858315 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:44.859791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:44.860673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:44.860673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.860673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:44.860673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.860673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.860673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:44.860673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.860673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:44.861820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.862780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:44.863665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.863665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:44.958648 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:44.959745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:44.960228 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:44.960633 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:44.960863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.961597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:44.961597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.961597 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:44.961597 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.961597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.961597 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:44.961597 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.961597 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.962345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:44.962931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:49.573657 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:49.575565 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:49.609838 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:49.610310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.611399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:49.611399 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.611399 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:49.611399 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.611399 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.611399 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:49.611399 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.611399 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.612946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.613502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:49.732347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:49.733520 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:49.733841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:49.734037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.734794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:49.734794 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.734794 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:49.734794 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.734794 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.734794 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:49.734794 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.734794 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.736517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.737019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:49.745528 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:49.745848 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:49.746522 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:49.746906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.747745 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:49.747745 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.747745 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:49.747745 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.747745 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.747745 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:49.747745 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.747745 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:49.748483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:49.748993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:49.748993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.748993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:49.748993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.748993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.748993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:49.748993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.748993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:49.749392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.749893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.750421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:49.750421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.750421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:49.856665 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:49.857084 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:49.857630 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:49.857979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.858609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:49.858609 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.858609 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:49.858609 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.858609 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.858609 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:49.858609 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.858609 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:49.859856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:49.860628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:49.860628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.860628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:49.860628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.860628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.860628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:49.860628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.860628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:49.861618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:49.862815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.862815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:49.960014 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:49.960614 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:49.961325 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:49.961779 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:49.962025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.962946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:49.962946 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.962946 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:49.962946 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.962946 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.962946 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:49.962946 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.962946 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.964682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:49.965575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:54.574433 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:54.576149 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:54.611378 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:54.612046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.612897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:54.612897 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.612897 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:54.612897 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.612897 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.612897 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:54.612897 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.612897 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.614054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.615002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:54.732845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:54.733819 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:54.734097 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:54.734361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.735048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:54.735048 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.735048 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:54.735048 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.735048 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.735048 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:54.735048 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.735048 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.736637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.737174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:54.745944 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:54.746350 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:54.746791 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:54.747045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.747738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:54.747738 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.747738 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:54.747738 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.747738 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.747738 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:54.747738 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.747738 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:54.748886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:54.749835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:54.749835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.749835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:54.749835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.749835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.749835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:54.749835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.749835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:54.750348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.751122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.752077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:54.752077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.752077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:54.857109 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:54.857588 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:54.858183 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:54.858588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.859369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:54.859369 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.859369 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:54.859369 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.859369 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.859369 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:54.859369 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.859369 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:54.860725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:54.861495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:54.861495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.861495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:54.861495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.861495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.861495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:54.861495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.861495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:54.862363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.863083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:54.863998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.863998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:54.960581 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:54.960945 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:54.961513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:54.961862 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:54.962084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.962945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:54.962945 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.962945 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:54.962945 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.962945 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.962945 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:54.962945 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.962945 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.964458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:54.965340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:59.575246 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:59.576928 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:59.610046 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:59.610418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.611337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:36:59.611337 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.611337 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:59.611337 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.611337 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.611337 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:59.611337 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.611337 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.612494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.613430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:59.733595 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:59.734894 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:59.735157 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:59.735398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.736126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:59.736126 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.736126 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:59.736126 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.736126 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.736126 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:59.736126 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.736126 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.737962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.738721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:59.746464 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:59.746824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:36:59.747225 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:59.747509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.748040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:59.748040 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.748040 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:59.748040 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.748040 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.748040 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:59.748040 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.748040 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:59.748642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:59.749125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:59.749125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.749125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:59.749125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.749125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.749125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:59.749125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.749125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:36:59.749438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.749898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.750430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:59.750430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.750430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:59.857753 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:59.858245 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:59.858977 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:59.859416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.860144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:59.860144 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.860144 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:59.860144 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.860144 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.860144 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:59.860144 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.860144 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:36:59.861453 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:59.862351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:36:59.862351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.862351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:36:59.862351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.862351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.862351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:36:59.862351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.862351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:36:59.863040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.863817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:59.864612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.864612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:36:59.961139 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:36:59.961698 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:36:59.962398 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:59.962771 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:36:59.963000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.963759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:36:59.963759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.963759 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:36:59.963759 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.963759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.963759 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:36:59.963759 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.963759 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.965460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:36:59.966267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:04.575964 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:04.577960 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:04.613672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:04.614103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.614890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:04.614890 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.614890 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:04.614890 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.614890 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.614890 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:04.614890 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.614890 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.617189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:04.734687 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:04.735750 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:04.736012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:04.736230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.737110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:04.737110 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.737110 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:04.737110 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.737110 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.737110 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:04.737110 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.737110 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.738447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.739242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:04.746893 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:04.747331 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:04.747798 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:04.748059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.748705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:04.748705 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.748705 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:04.748705 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.748705 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.748705 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:04.748705 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.748705 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:04.749641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:04.750126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:04.750126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.750126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:04.750126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.750126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.750126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:04.750126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.750126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:04.750579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.751009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.751528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:04.751528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.751528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:04.858362 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:04.858642 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:04.859368 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:04.859837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.860772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:04.860772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.860772 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:04.860772 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.860772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.860772 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:04.860772 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.860772 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:04.861192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:04.861874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:04.861874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.861874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:04.861874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.861874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.861874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:04.861874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.861874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:04.862268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.862857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:04.863468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.863468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:04.961888 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:04.962633 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:04.963265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:04.963631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:04.963879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.964634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:04.964634 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.964634 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:04.964634 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.964634 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.964634 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:04.964634 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.964634 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.966579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:04.967486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:09.576741 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:09.578412 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:09.618282 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:09.618504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.619068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:09.619068 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.619068 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:09.619068 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.619068 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.619068 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:09.619068 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.619068 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.620119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.621728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:09.735474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:09.737140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:09.737140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:09.737660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:09.738366 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.738366 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:09.738366 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.738366 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.738366 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:09.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.740340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.741341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:09.747275 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:09.747634 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:09.748058 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:09.748266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.748805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:09.748805 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.748805 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:09.748805 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.748805 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.748805 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:09.748805 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.748805 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:09.750224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:09.750659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:09.750659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.750659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:09.750659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.750659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.750659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:09.750659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.750659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:09.751862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.752336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.752823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:09.752823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.752823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:09.859019 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:09.859467 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:09.859748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:09.860112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.860907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:09.860907 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.860907 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:09.860907 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.860907 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.860907 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:09.860907 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.860907 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:09.862373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:09.863172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:09.863172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.863172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:09.863172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.863172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.863172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:09.863172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.863172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:09.863926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.864802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:09.865870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.865870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:09.962506 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:09.962907 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:09.963475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:09.963842 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:09.964078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.964825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:09.964825 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.964825 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:09.964825 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.964825 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.964825 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:09.964825 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.964825 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.966428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:09.967170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:14.577315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:14.578909 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:14.610657 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:14.610957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.611572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:14.611572 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.611572 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:14.611572 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.611572 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.611572 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:14.611572 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.611572 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.612870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.613744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:14.737240 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:14.737918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:14.738485 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:14.738485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.739254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:14.739254 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.739254 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:14.739254 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.739254 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.739254 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:14.739254 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.739254 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.740824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.741783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:14.747603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:14.747962 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:14.748293 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:14.748477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.748889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:14.748889 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.748889 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:14.748889 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.748889 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.748889 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:14.748889 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.748889 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:14.749614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:14.749978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:14.749978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.749978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:14.749978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.749978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.749978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:14.749978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.749978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:14.750608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.750984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.751501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:14.751501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.751501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:14.859482 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:14.860046 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:14.860700 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:14.861090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.861850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:14.861850 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.861850 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:14.861850 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.861850 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.861850 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:14.861850 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.861850 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:14.862968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:14.863751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:14.863751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.863751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:14.863751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.863751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.863751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:14.863751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.863751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:14.864450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.865205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:14.866096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.866096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:14.963090 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:14.963682 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:14.964203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:14.964628 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:14.964844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.965611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:14.965611 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.965611 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:14.965611 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.965611 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.965611 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:14.965611 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.965611 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.967433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:14.968577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:19.578121 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:19.579840 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:19.623114 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:19.623513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.624385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:19.624385 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.624385 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:19.624385 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.624385 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.624385 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:19.624385 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.624385 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.625542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.626405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:19.737135 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:19.738325 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:19.738667 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:19.738898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.739576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:19.739576 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.739576 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:19.739576 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.739576 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.739576 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:19.739576 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.739576 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.740724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.741188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:19.748037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:19.748477 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:19.748939 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:19.749243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.749913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:19.749913 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.749913 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:19.749913 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.749913 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.749913 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:19.749913 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.749913 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:19.751143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:19.751924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:19.751924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.751924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:19.751924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.751924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.751924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:19.751924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.751924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:19.752634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.753350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.754145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:19.754145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.754145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:19.860198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:19.860707 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:19.861310 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:19.861703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.862504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:19.862504 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.862504 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:19.862504 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.862504 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.862504 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:19.862504 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.862504 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:19.863804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:19.864492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:19.864492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.864492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:19.864492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.864492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.864492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:19.864492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.864492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:19.865410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.866161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:19.866984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.866984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:19.963666 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:19.964146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:19.964735 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:19.965132 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:19.965485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.966168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:19.966168 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.966168 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:19.966168 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.966168 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.966168 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:19.966168 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.966168 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.967094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:19.967909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:24.578901 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:24.581019 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:24.611100 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:24.611903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.612245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:24.612245 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.612245 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:24.612245 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.612245 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.612245 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:24.612245 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.612245 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.613091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.614037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:24.753779 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:24.753779 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:24.753779 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:24.754486 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:24.754486 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:24.754964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.755795 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:24.756008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.756657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:24.756657 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.756657 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:24.756657 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.756657 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.756657 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:24.756657 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.756657 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.757928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:24.758674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:24.759438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.760278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:24.760972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.761472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.761997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:24.761997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.761997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:24.860853 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:24.861435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:24.862204 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:24.862646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.863506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Traceback (most recent call last):
Mar 23 22:37:24.863506 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.863506 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:24.863506 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.863506 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.863506 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:24.863506 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.863506 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:24.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:24.865865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:24.865865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:24.865865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.865865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:24.865865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.865865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.865865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:24.865865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.865865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:24.866901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.867779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:24.868662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.868662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:24.964266 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:24.964763 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:24.965454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:24.965902 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:24.966121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.967024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:24.967024 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.967024 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:24.967024 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.967024 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.967024 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:24.967024 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.967024 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.968790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:24.969332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:29.579683 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:29.580788 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:29.611107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:29.611522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.612402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:29.612402 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.612402 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:29.612402 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.612402 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.612402 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:29.612402 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.612402 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.613558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.614053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:29.753802 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:29.754026 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:29.755382 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:29.756299 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:29.756718 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:29.756966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.758039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:37:29.758039 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.758039 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:29.758039 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.758039 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.758039 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:29.758039 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.758039 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:29.760149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.761922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:29.762477 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:29.762477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.762477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:29.762477 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.762477 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:29.762477 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.762477 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.762477 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:29.762477 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.762477 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:29.763031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:29.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:29.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:29.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:29.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.763561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:29.764579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.765043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.765637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:29.765637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.765637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:29.861748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:29.862378 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:29.863208 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:29.863649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.864542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:29.864542 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.864542 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:29.864542 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.864542 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.864542 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:29.864542 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.864542 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:29.866102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:29.867107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:29.867107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.867107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:29.867107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.867107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.867107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:29.867107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.867107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:29.867619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.868667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:29.869653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.869653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:29.965039 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:29.965781 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:29.966370 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:29.966656 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:29.966805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.967492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:29.967492 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.967492 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:29.967492 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.967492 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.967492 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:29.967492 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.967492 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.969502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:29.969869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:34.580381 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:34.581620 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:34.595240 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=239 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:37:34.617167 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:34.617558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.618338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:34.618338 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.618338 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:34.618338 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.618338 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.618338 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:34.618338 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.618338 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.619319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.620364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:34.754849 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:34.755912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:34.756467 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:34.757084 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:34.757454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:34.757698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.758539 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:34.758821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.759744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:34.759744 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.759744 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:34.759744 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.759744 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.759744 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:34.759744 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.759744 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.761658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:34.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:34.763261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.764053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:34.764652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.765342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.766142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:34.766142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.766142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:34.862505 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:34.863141 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:34.863889 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:34.864330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.865164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:34.865164 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.865164 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:34.865164 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.865164 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.865164 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:34.865164 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.865164 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:34.866551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:34.867443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:34.867443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.867443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:34.867443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.867443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.867443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:34.867443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.867443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:34.868155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.868960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:34.869954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.869954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:34.965702 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:34.966485 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:34.967135 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:34.967641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:34.967899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.968757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:34.968757 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.968757 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:34.968757 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.968757 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.968757 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:34.968757 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.968757 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.970434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:34.971329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:39.357551 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:39.366242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:389}}
Mar 23 22:37:39.366401 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=104232) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:370}}
Mar 23 22:37:39.366526 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Fixing resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:37:39.401622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, resource={'id': '06cffee1-b8de-433e-a67a-ec0e950c9238', 'name': 'private', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['130adef4-306b-4397-a096-91ee5094d0e2', 'fb1912da-e258-4d41-8cd4-7896f0de6385'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:17:25Z', 'updated_at': '2026-03-23T22:17:28Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 36968}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.402031 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.402611 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c4286b-d53b-4217-a9f3-05177afa9297, external_ids_update=None, columns={'tag': 36968}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.402811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.403798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:37:39.403798 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.403798 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:39.403798 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.403798 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.403798 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:39.403798 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.403798 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:39.405039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Failed to fix resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.405039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:37:39.405039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:37:39.405039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:37:39.405039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:37:39.405039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:37:39.405039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:37:39.405039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:39.405039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.405039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.405039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.405679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.406440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:37:39.406440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.406440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:37:39.406440 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Fixing resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:37:39.433109 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, resource={'id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'name': 'public', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'a53179c9-417a-4560-9bc7-e3f64551b15b'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-23T22:17:31Z', 'updated_at': '2026-03-23T22:17:40Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.433528 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.434136 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a8bd993e-d23e-4e73-8066-766307763bdb, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.434500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.435380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:37:39.435380 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.435380 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:39.435380 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.435380 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.435380 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:39.435380 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.435380 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:39.437179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Failed to fix resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.437179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:37:39.437179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:37:39.437179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:37:39.437179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:37:39.437179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:37:39.437179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:37:39.437179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:39.437179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.437179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.437179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.437671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.438180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:37:39.438180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.438180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:37:39.438180 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Fixing resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:37:39.460500 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, resource={'id': '3a042d54-7a79-465e-bd57-47cd6833a4d0', 'name': 'admin_net', 'tenant_id': 'b4ffb2af173e4612a30978ef510f4f44', 'project_id': 'b4ffb2af173e4612a30978ef510f4f44', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['935d20c1-7d6b-489a-8358-b5fb94945b7e'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:20:57Z', 'updated_at': '2026-03-23T22:20:58Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 47875}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.460923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.461517 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c442d0-efff-4c7e-bcb6-377b4da39c34, external_ids_update=None, columns={'tag': 47875}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.461726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.462593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:37:39.462593 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.462593 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:39.462593 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.462593 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.462593 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:39.462593 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.462593 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:39.463888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Failed to fix resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.463888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:37:39.463888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:37:39.463888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:37:39.463888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:37:39.463888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:37:39.463888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:37:39.463888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:39.463888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.463888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.463888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.464471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.465059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:37:39.465059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.465059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:37:39.465059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Fixing resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:37:39.490565 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, resource={'id': 'c9e6c30a-b30e-473b-b06a-50be75c90b36', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['421749b7-bb0d-4854-b3f4-f0414f4c37fb'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:21:34Z', 'updated_at': '2026-03-23T22:21:35Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 48210}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.490943 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.491536 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13535482-a705-4453-af87-e8a7d8c10c82, external_ids_update=None, columns={'tag': 48210}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.491732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.492544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:37:39.492544 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.492544 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:39.492544 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.492544 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.492544 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:39.492544 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.492544 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:39.493454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Failed to fix resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.493454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:37:39.493454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:37:39.493454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:37:39.493454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:37:39.493454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:37:39.493454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:37:39.493454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:39.493454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.493454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.493454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.493825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.494169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:37:39.494169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.494169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:37:39.494169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Fixing resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:37:39.568237 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, resource={'id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'enable_snat': True}, 'gw_port_id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-t62km', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'ip_address': '172.24.5.146', 'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a'}, {'ip_address': '2001:db8::396', 'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 3, 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d'}, resource_type=routers, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.568802 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.569222 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.569513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.569718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.570753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:37:39.570753 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.570753 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:39.570753 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.570753 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.570753 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:39.570753 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.570753 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:39.571120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Unable to update router 154c08d6-78c5-49fd-a785-0df3f4277a23. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.572734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Failed to fix resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.572734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:37:39.572734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:37:39.572734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:37:39.572734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:37:39.572734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:37:39.572734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 23 22:37:39.572734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 23 22:37:39.572734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 23 22:37:39.572734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 23 22:37:39.572734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:37:39.573354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:37:39.573906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.573906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:37:39.573906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.573906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:37:39.573906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.573906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.573906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:37:39.573906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.573906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:37:39.573906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Fixing resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:37:39.581146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:39.586585 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:39.640120 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.640596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.641518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:39.641518 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.641518 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:39.641518 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.641518 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.641518 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:39.641518 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.641518 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.644324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.645371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:39.716124 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, resource={'id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'name': '', 'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'project_id': '', 'mac_address': 'fa:16:3e:aa:0e:57', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160382', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.716437 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a a53179c9-417a-4560-9bc7-e3f64551b15b', 'neutron:network_name': 'neutron-bb9d5c47-4694-42c2-a018-18e3c9058915', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-154c08d6-78c5-49fd-a785-0df3f4277a23'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::396/64', '172.24.5.146/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.716877 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=348aba7c-88a8-4a19-9486-df3f7edb4007, lrouter_port=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.717131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.717926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:37:39.717926 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.717926 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:39.717926 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.717926 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.717926 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:39.717926 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.717926 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:39.719452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Failed to fix resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.719452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:37:39.719452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:37:39.719452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:37:39.719452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:37:39.719452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:37:39.719452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 23 22:37:39.719452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:39.719452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.719452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.719452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.719906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.720275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:37:39.720275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.720275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:37:39.720275 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Maintenance task: Synchronization completed (took 0.35 seconds)
Mar 23 22:37:39.755431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:39.756435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:39.757286 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:39.757890 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.758188 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.758473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.759542 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.759836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.760871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:39.760871 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.760871 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:39.760871 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.760871 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.760871 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:39.760871 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.760871 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.762039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:39.762919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:37:39.763817 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.763817 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:39.763817 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.763817 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.763817 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:39.763817 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.763817 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:39.764147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:39.764999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.764999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:39.764999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.764999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:39.764999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.764999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.764999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:39.764999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.764999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:39.765797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.766395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.767377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:39.767377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.767377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:39.863184 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:39.863746 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:39.864366 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.864843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.865652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:39.865652 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.865652 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:39.865652 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.865652 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.865652 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:39.865652 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.865652 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:39.866876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.868167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:39.868167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.868167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:39.868167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.868167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.868167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:39.868167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.868167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:39.868712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.869524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:39.870044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.870044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:39.966523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:39.966861 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:39.967535 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.967918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:39.968161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.969043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:39.969043 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.969043 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:39.969043 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.969043 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.969043 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:39.969043 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.969043 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.970783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:39.971603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:44.581985 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:44.584596 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:44.621670 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:44.622175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.623248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:44.623248 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.623248 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:44.623248 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.623248 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.623248 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:44.623248 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.623248 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.624401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.625375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:44.756176 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:44.757208 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:44.757938 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:44.758816 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:44.759238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:44.759458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.760277 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:44.760627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.761267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:37:44.761267 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.761267 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:44.761267 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.761267 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.761267 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:44.761267 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.761267 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:44.763049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.763996 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:44.765239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.766036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.766874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:44.766874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.766874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.767983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:44.864036 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:44.864505 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:44.865798 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:44.866349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.867423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:44.867423 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.867423 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:44.867423 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.867423 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.867423 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:44.867423 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.867423 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:44.868799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:44.869565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:44.869565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.869565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:44.869565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.869565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.869565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:44.869565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.869565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:44.870584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.871660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:44.872669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.872669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:44.967007 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:44.967490 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:44.968068 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:44.968573 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:44.968818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.969593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:44.969593 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.969593 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:44.969593 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.969593 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.969593 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:44.969593 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.969593 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.971286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:44.972296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:49.582846 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:49.584401 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:49.617047 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:49.617470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.618314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:49.618314 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.618314 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:49.618314 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.618314 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.618314 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:49.618314 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.618314 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.619153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.620198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:49.756943 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:49.760165 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:49.760520 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:49.760666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.762601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:37:49.762601 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.762601 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:49.762601 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.762601 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.762601 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:49.762601 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.762601 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:49.766153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.767034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:49.768127 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:49.768127 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:49.768127 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:49.768520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.769428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:37:49.769428 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.769428 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:49.769428 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.769428 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.769428 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:49.769428 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.769428 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:49.770606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:49.771153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:49.771153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:49.771153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.771153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:49.771153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.771153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.771153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:49.771153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.771153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:49.771909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.773088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.773630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:49.773630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.773630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:49.864735 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:49.865051 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:49.865683 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:49.866046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.866858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:49.866858 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.866858 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:49.866858 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.866858 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.866858 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:49.866858 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.866858 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:49.868262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:49.869059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:49.869059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.869059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:49.869059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.869059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.869059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:49.869059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.869059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:49.870028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.871000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:49.872333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.872333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:49.967518 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:49.968083 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:49.968810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:49.969244 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:49.969491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.970386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:49.970386 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.970386 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:49.970386 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.970386 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.970386 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:49.970386 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.970386 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.972163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:49.973054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:54.583547 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:54.585313 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:54.615673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:54.615830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:54.616315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.616315 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:54.616315 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.616315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.616315 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:54.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.616315 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.617436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.618376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:54.757813 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:54.758923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:54.759210 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:54.759467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.760188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:54.760188 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.760188 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:54.760188 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.760188 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.760188 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:54.760188 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.760188 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.762019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.763120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:54.767394 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:54.767777 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:54.768305 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:54.768831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.769279 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:54.769279 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.769279 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:54.769279 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.769279 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.769279 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:54.769279 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.769279 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:54.769987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:54.770489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:54.770489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.770489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:54.770489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.770489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.770489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:54.770489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.770489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:54.770884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.771326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.771786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:54.771786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.771786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:54.872032 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:54.872609 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:54.873579 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:54.873909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.874754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:54.874754 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.874754 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:54.874754 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.874754 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.874754 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:54.874754 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.874754 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.876564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:54.877317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:54.878163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:54.878163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:54.878163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.878163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:54.878163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.878163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.878163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:54.878163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.878163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:54.967960 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:54.968590 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:54.969668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:54.970068 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:54.970358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.971200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:54.971200 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.971200 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:54.971200 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.971200 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.971200 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:54.971200 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.971200 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.972894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:54.973517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:59.584526 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:59.586426 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:59.616843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:59.617293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.618159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:37:59.618159 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.618159 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:59.618159 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.618159 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.618159 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:59.618159 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.618159 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.619341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.620325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:59.758602 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:59.759866 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:59.760230 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:59.760520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.761459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:59.761459 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.761459 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:59.761459 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.761459 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.761459 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:59.761459 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.761459 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.770444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.771362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:59.772146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:59.772146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:37:59.772146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:59.772146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.772862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:59.772862 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.772862 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:59.772862 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.772862 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.772862 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:59.772862 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.772862 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:59.773415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.774319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:59.775117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:59.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:59.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:59.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:59.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:59.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:59.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:59.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:59.872695 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:59.873388 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:59.874049 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:59.874374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.874986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Traceback (most recent call last):
Mar 23 22:37:59.874986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.874986 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:59.874986 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.874986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.874986 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:59.874986 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.874986 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:37:59.876295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:37:59.876729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:59.876729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:37:59.876729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.876729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:37:59.876729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.876729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.876729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:37:59.876729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.876729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-2034fd20-17e9-40fe-a188-3ec1678f7e47 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:37:59.877547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.878011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:59.878526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.878526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:37:59.968565 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:37:59.969208 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:37:59.969650 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:59.970105 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:37:59.970105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.970795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:37:59.970795 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.970795 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:37:59.970795 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.970795 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.970795 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:37:59.970795 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.970795 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.972590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:37:59.973415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:04.585323 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:04.586958 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:04.622572 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:04.623017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.623914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:04.623914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.623914 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:04.623914 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.623914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.623914 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:04.623914 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.623914 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.625125 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.625873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:04.759062 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:04.760237 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:04.760618 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:04.760853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.761668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:04.761668 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.761668 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:04.761668 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.761668 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.761668 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:04.761668 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.761668 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.762822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.763709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:04.771111 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:04.771604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:04.772130 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:04.772504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:04.773245 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.773245 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:04.773245 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.773245 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.773245 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:04.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.773245 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:04.774629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:04.775145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:04.775145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.775145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:04.775145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.775145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.775145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:04.775145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.775145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:04.776111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.776615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.777140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:04.777140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.777140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:04.873474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:04.873828 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:04.874627 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:04.875092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.876027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:04.876027 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.876027 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:04.876027 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.876027 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.876027 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:04.876027 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.876027 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:04.877692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:04.878466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:04.878466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.878466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:04.878466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.878466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.878466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:04.878466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.878466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:04.880090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.880883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:04.881700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.881700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:04.969203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:04.969824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:04.970382 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:04.970776 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:04.971010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.971802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:04.971802 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.971802 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:04.971802 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.971802 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.971802 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:04.971802 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.971802 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.973570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:04.974414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:09.585858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:09.587465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:09.626850 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:09.627254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.628126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:09.628126 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.628126 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:09.628126 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.628126 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.628126 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:09.628126 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.628126 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.629310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.630321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:09.759568 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:09.761002 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:09.761408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:09.761700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:09.762441 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.762441 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:09.762441 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.762441 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.762441 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:09.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.762441 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.764088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.764975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:09.771407 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:09.771582 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:09.772043 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:09.772385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.773050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:09.773050 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.773050 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:09.773050 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.773050 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.773050 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:09.773050 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.773050 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:09.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:09.774249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:09.774249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.774249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:09.774249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.774249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.774249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:09.774249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.774249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:09.774651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.775190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.775698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:09.775698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.775698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:09.874541 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:09.875015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:09.875538 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:09.875904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.876720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:09.876720 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.876720 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:09.876720 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.876720 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.876720 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:09.876720 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.876720 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:09.878094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:09.879008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:09.879008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.879008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:09.879008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.879008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.879008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:09.879008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.879008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:09.879755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.880808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:09.881700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.881700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:09.969872 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:09.970420 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:09.970857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:09.971238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:09.971506 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.972280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:09.972280 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.972280 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:09.972280 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.972280 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.972280 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:09.972280 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.972280 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.973905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:09.974736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:14.586478 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:14.587968 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:14.624991 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:14.625560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.626540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:14.626540 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.626540 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:14.626540 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.626540 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.626540 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:14.626540 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.626540 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.627684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.628540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:14.760068 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:14.761006 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:14.761393 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:14.761637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.762489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:14.762489 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.762489 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:14.762489 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.762489 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.762489 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:14.762489 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.762489 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.764393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.765309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:14.771725 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:14.772107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:14.772481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:14.772661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.773116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:14.773116 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.773116 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:14.773116 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.773116 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.773116 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:14.773116 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.773116 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:14.773817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:14.774325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:14.774325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.774325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:14.774325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.774325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.774325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:14.774325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.774325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:14.774747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.775377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.775858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:14.775858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.775858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:14.875416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:14.875843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:14.876540 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:14.876985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.877943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:38:14.877943 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.877943 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:14.877943 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.877943 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.877943 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:14.877943 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.877943 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:14.879617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:14.880403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:14.880403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:14.880403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.880403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:14.880403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.880403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.880403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:14.880403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.880403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:14.880834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.881549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:14.882714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.882714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:14.970465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:14.970812 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:14.971300 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:14.971612 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:14.971905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.972694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:14.972694 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.972694 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:14.972694 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.972694 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.972694 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:14.972694 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.972694 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.973831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:14.974593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:19.587249 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:19.588564 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:19.622080 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:19.622282 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.623085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:19.623085 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.623085 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:19.623085 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.623085 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.623085 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:19.623085 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.623085 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.624257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.624823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:19.760837 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:19.762318 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:19.763038 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:19.763038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.763675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:19.763675 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.763675 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:19.763675 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.763675 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.763675 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:19.763675 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.763675 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.765703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.766644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:19.772383 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:19.772824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:19.773274 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:19.773567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.774099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:19.774099 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.774099 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:19.774099 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.774099 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.774099 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:19.774099 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.774099 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:19.774909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:19.775480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:19.775480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.775480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:19.775480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.775480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.775480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:19.775480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.775480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:19.775969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.776456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.777003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:19.777003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.777003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:19.876029 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:19.876586 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:19.877492 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:19.878731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.879673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:38:19.879673 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.879673 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:19.879673 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.879673 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.879673 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:19.879673 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.879673 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:19.880597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:19.881616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:19.881616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:19.881616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.881616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:19.881616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.881616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.881616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:19.881616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.881616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:19.882251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.883425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:19.884624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.884624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:19.971048 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:19.971895 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:19.972613 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:19.973086 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:19.973406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.974231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:19.974231 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.974231 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:19.974231 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.974231 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.974231 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:19.974231 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.974231 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.975871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:19.976383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:24.587992 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:24.589470 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:24.624586 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:24.624897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.625868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:24.625868 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.625868 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:24.625868 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.625868 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.625868 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:24.625868 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.625868 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.627407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.628484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:24.761770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:24.762910 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:24.763354 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:24.763682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.764515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:24.764515 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.764515 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:24.764515 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.764515 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.764515 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:24.764515 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.764515 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.765856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.766765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:24.773293 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:24.773858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:24.774512 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:24.774797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.775599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:24.775599 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.775599 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:24.775599 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.775599 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.775599 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:24.775599 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.775599 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:24.776885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:24.777861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:24.777861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.777861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:24.777861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.777861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.777861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:24.777861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.777861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:24.778327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.779120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.780044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:24.780044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.780044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:24.876909 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:24.877732 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:24.878473 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:24.878832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.879771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:38:24.879771 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.879771 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:24.879771 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.879771 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.879771 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:24.879771 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.879771 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:24.881250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:24.881912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:24.881912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:24.881912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.881912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:24.881912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.881912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.881912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:24.881912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.881912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:24.882343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.883053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:24.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:24.971849 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:24.972514 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:24.972980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:24.973239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:24.973367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.974033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:24.974033 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.974033 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:24.974033 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.974033 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.974033 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:24.974033 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.974033 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.976262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:24.977072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:29.588678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:29.590389 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:29.625072 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:29.625603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.626917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:29.626917 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.626917 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:29.626917 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.626917 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.626917 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:29.626917 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.626917 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.628137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.628650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:29.762637 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:29.764651 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:29.765072 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:29.765404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.766677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:29.766677 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.766677 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:29.766677 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.766677 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.766677 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:29.766677 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.766677 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.768753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.769753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:29.774082 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:29.774634 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:29.775226 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:29.775587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.776404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:38:29.776404 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.776404 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:29.776404 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.776404 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.776404 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:29.776404 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.776404 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:29.777361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:29.777822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:29.777822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:29.777822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.777822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:29.777822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.777822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.777822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:29.777822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.777822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:29.778564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.780010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:29.780010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.780010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:29.877751 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:29.878346 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:29.879129 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:29.879590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.880536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:38:29.880536 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.880536 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:29.880536 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.880536 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.880536 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:29.880536 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.880536 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:29.881602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:29.882652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:29.882652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:29.882652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.882652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:29.882652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.882652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.882652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:29.882652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.882652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:29.883114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.884183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:29.885319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.885319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:29.972567 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:29.973146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:29.973752 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:29.974118 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:29.974411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.975165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:29.975165 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.975165 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:29.975165 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.975165 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.975165 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:29.975165 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.975165 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.983429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:29.984304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:34.589456 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:34.590992 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:34.604050 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=434 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:38:34.621982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:34.622403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.623230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:34.623230 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.623230 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:34.623230 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.623230 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.623230 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:34.623230 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.623230 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.624517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.625000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:34.763465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:34.764974 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:34.765435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:34.765696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.766758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:34.766758 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.766758 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:34.766758 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.766758 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.766758 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:34.766758 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.766758 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.767757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.768257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:34.774604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:34.775188 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:34.775781 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:34.776023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.776722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:38:34.776722 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.776722 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:34.776722 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.776722 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.776722 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:34.776722 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.776722 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:34.778129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:34.778888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:34.778888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:34.778888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.778888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:34.778888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.778888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.778888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:34.778888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.778888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:34.779746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.780525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.781415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:34.781415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.781415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:34.878478 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:34.879164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:34.880536 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:34.881042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.882339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:38:34.882339 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.882339 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:34.882339 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.882339 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.882339 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:34.882339 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.882339 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:34.883876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:34.884611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:34.884611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:34.884611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.884611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:34.884611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.884611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.884611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:34.884611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.884611 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:34.886208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.886995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:34.887648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.887648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:34.973350 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:34.973796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:34.975071 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:34.975784 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:34.976069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.977098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:34.977098 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.977098 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:34.977098 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.977098 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.977098 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:34.977098 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.977098 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.978740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:34.979314 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:39.590059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:39.591281 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:39.625668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:39.626095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.627178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:39.627178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.627178 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:39.627178 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.627178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.627178 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:39.627178 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.627178 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.628146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.628978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:39.764182 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:39.765784 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:39.766106 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:39.766307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:39.767207 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.767207 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:39.767207 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.767207 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.767207 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:39.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.769571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.770413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:39.775036 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:39.776032 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:39.776831 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:39.777186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.777901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:38:39.777901 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.777901 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:39.777901 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.777901 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.777901 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:39.777901 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.777901 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:39.778710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:39.779447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:39.779447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:39.779447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.779447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:39.779447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.779447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.779447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:39.779447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.779447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:39.779789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.780310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.781818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:39.781818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.781818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:39.879256 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:39.879810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:39.880972 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:39.881526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.882845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:38:39.882845 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.882845 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:39.882845 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.882845 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.882845 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:39.882845 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.882845 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:39.884422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:39.885186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:39.885186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:39.885186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.885186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:39.885186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.885186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.885186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:39.885186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.885186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:39.886158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.886976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:39.888031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.888031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:39.974121 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:39.975180 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:39.976429 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:39.976981 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:39.977487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.978751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:38:39.978751 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.978751 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:39.978751 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.978751 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.978751 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:39.978751 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.978751 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.980698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:39.981632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:44.590652 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:44.591859 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:44.621060 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:44.621060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.621868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:44.621868 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.621868 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:44.621868 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.621868 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.621868 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:44.621868 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.621868 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.622608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.623202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:44.764868 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:44.766172 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:44.766531 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:44.766764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.767629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:44.767629 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.767629 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:44.767629 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.767629 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.767629 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:44.767629 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.767629 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.769507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.770303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:44.775530 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:44.776088 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:44.776693 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:44.776942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.777762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:38:44.777762 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.777762 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:44.777762 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.777762 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.777762 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:44.777762 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.777762 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:44.779094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:44.779980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:44.779980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:44.779980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.779980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:44.779980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.779980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.779980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:44.779980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.779980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:44.780658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.781472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.782307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:44.782307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.782307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:44.879764 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:44.880296 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:44.880840 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:44.881208 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.881991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:38:44.881991 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.881991 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:44.881991 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.881991 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.881991 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:44.881991 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.881991 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:44.883289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:44.883960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:44.883960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:44.883960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.883960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:44.883960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.883960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.883960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:44.883960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.883960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:44.884902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.885765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:44.886587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.886587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:44.975077 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:44.975632 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:44.976188 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:44.976569 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:44.976799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.977702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:44.977702 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.977702 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:44.977702 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.977702 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.977702 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:44.977702 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.977702 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.979412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:44.980318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:49.591299 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:49.592827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:49.626838 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:49.627372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.628386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:49.628386 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.628386 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:49.628386 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.628386 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.628386 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:49.628386 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.628386 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.629445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.630897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:49.765568 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:49.775083 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:49.775410 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:49.775516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.776258 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:49.776258 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.776258 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:49.776258 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.776258 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.776258 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:49.776258 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.776258 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.778491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.778996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:49.779480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:49.779480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:49.779950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:49.780239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.780876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:49.780876 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.780876 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:49.780876 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.780876 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.780876 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:49.780876 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.780876 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:49.782182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:49.782747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:49.782747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.782747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:49.782747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.782747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.782747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:49.782747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.782747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:49.783877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.784360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.784840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:49.784840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.784840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:49.880447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:49.880961 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:49.881809 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:49.882283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.883295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:38:49.883295 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.883295 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:49.883295 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.883295 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.883295 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:49.883295 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.883295 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:49.884623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:49.885048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:49.885048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:49.885048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.885048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:49.885048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.885048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.885048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:49.885048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.885048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:49.886309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.886757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:49.887265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.887265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:49.975704 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:49.976455 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:49.977086 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:49.977404 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:49.977529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.978246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:38:49.978246 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.978246 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:49.978246 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.978246 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.978246 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:49.978246 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.978246 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.979895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:49.980796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:54.591853 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:54.593667 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:54.631147 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:54.631613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.632488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:54.632488 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.632488 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:54.632488 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.632488 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.632488 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:54.632488 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.632488 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.633402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.634418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:54.766353 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:54.767439 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:54.767774 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:54.768032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.768927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:38:54.768927 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.768927 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:54.768927 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.768927 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.768927 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:54.768927 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.768927 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:54.770055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.770875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:54.779448 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:54.779869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:54.780398 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:54.780662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.781451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:54.781451 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.781451 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:54.781451 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.781451 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.781451 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:54.781451 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.781451 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:54.783034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:54.784428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:54.784428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.784428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:54.784428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.784428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.784428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:54.784428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.784428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:54.785706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.786690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.787552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:54.787552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.787552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:54.880975 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:54.881396 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:54.882025 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:54.882418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.883167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:38:54.883167 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.883167 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:54.883167 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.883167 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.883167 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:54.883167 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.883167 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:54.884606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:54.885285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:54.885285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:54.885285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.885285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:54.885285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.885285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.885285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:54.885285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.885285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:54.886310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.887027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:54.887865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.887865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:54.976364 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:54.976930 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:54.977589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:54.977941 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:54.978180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.979087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:38:54.979087 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.979087 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:54.979087 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.979087 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.979087 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:54.979087 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.979087 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.980867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:54.981723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:59.592717 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:59.594474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:59.624239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:59.624482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.625179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:38:59.625179 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.625179 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:59.625179 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.625179 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.625179 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:59.625179 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.625179 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.627131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.628118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:59.767030 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:59.768462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:59.768804 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:59.769052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.770018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:38:59.770018 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.770018 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:59.770018 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.770018 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.770018 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:59.770018 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.770018 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:59.771309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.772584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:59.779872 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:59.780469 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:38:59.781006 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:59.781334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.782057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:38:59.782057 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.782057 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:59.782057 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.782057 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.782057 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:59.782057 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.782057 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:59.783434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:59.783892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:59.783892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.783892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:59.783892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.783892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.783892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:59.783892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.783892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:38:59.785040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.785490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.786045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:59.786045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.786045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:59.881604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:59.882184 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:59.882863 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:59.883192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.883926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:38:59.883926 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.883926 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:59.883926 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.883926 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.883926 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:59.883926 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.883926 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:38:59.885435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:38:59.886607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:59.886607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:38:59.886607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.886607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:38:59.886607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.886607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.886607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:38:59.886607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.886607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:38:59.887591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.888378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:59.889363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.889363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:38:59.976929 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:38:59.977712 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:38:59.978411 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:59.978905 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:38:59.979178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.980257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:38:59.980257 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.980257 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:38:59.980257 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.980257 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.980257 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:38:59.980257 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.980257 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.982053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:38:59.982879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:04.593707 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:04.595865 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:04.631602 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:04.631852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.632395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:04.632395 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.632395 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:04.632395 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.632395 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.632395 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:04.632395 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.632395 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.633550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.634824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:04.767683 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:04.769003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:04.769263 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:04.769591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.771073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:04.771073 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.771073 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:04.771073 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.771073 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.771073 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:04.771073 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.771073 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:04.772508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.773207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:04.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:04.780749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:04.781317 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:04.781622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.782341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:04.782341 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.782341 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:04.782341 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.782341 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.782341 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:04.782341 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.782341 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:04.783635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:04.784505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:04.784505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.784505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:04.784505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.784505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.784505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:04.784505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.784505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:04.785199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.785920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.786921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:04.786921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.786921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:04.882156 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:04.882529 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:04.883062 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:04.883515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.884325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:39:04.884325 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.884325 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:04.884325 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.884325 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.884325 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:04.884325 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.884325 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:04.885321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:04.885917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:04.885917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:04.885917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.885917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:04.885917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.885917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.885917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:04.885917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.885917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:04.886180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.886677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:04.887185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.887185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:04.977426 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:04.978053 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:04.978756 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:04.979387 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:04.979698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.980680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:39:04.980680 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.980680 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:04.980680 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.980680 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.980680 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:04.980680 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.980680 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.982393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:04.983165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:09.594537 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:09.596370 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:09.634045 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:09.634446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.635401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:09.635401 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.635401 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:09.635401 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.635401 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.635401 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:09.635401 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.635401 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.636613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.637666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:09.768268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:09.769471 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:09.769892 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:09.770171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.771091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:39:09.771091 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.771091 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:09.771091 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.771091 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.771091 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:09.771091 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.771091 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:09.772491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.773337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:09.780904 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:09.781415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:09.781970 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:09.782292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.783042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:09.783042 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.783042 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:09.783042 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.783042 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.783042 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:09.783042 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.783042 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:09.784395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:09.785270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:09.785270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:09.785270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.785270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:09.785270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.785270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.785270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:09.785270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.785270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:09.785959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.786818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.787684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:09.787684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.787684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:09.882830 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:09.883032 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:09.884437 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:09.884952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.885839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:39:09.885839 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.885839 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:09.885839 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.885839 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.885839 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:09.885839 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.885839 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:09.887133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:09.887824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:09.887824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:09.887824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.887824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:09.887824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.887824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.887824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:09.887824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.887824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:09.888832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.889630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:09.890440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.890440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:09.977979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:09.978752 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:09.980069 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:09.980631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:09.980901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.982135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Traceback (most recent call last):
Mar 23 22:39:09.982135 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.982135 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:09.982135 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.982135 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.982135 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:09.982135 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.982135 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d34d21f9-8a5d-40fc-bec2-51bcc6e3d700 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.983937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:09.984594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:14.595353 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:14.596745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:14.628895 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:14.629234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.630082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:14.630082 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.630082 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:14.630082 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.630082 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.630082 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:14.630082 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.630082 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.631148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.632072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:14.768898 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:14.770405 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:14.770625 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:14.770839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.771658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:39:14.771658 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.771658 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:14.771658 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.771658 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.771658 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:14.771658 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.771658 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:14.773747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.774384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:14.781634 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:14.782143 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:14.782588 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:14.782788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.783380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:14.783380 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.783380 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:14.783380 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.783380 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.783380 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:14.783380 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.783380 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:14.784676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:14.785544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:14.785544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:14.785544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.785544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:14.785544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.785544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.785544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:14.785544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.785544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:14.786268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.787288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.788160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:14.788160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.788160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:14.883387 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:14.883779 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:14.884363 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:14.884727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.885535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Traceback (most recent call last):
Mar 23 22:39:14.885535 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.885535 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:14.885535 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.885535 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.885535 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:14.885535 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.885535 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:14.895910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:14.896667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:14.896667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:14.896667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.896667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:14.896667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.896667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.896667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:14.896667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.896667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-d49914fb-8453-4ae1-b8d5-d016ed0e511c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:14.897781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.898555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:14.899476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.899476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:14.978521 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:14.979198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:14.979658 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:14.980014 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:14.980252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.981119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:14.981119 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.981119 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:14.981119 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.981119 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.981119 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:14.981119 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.981119 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.983088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:14.983959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:19.595880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:19.597243 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:19.635318 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:19.635834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.636993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:19.636993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.636993 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:19.636993 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.636993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.636993 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:19.636993 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.636993 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.638136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.638696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:19.769692 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:19.770828 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:19.771078 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:19.771204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.771954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:39:19.771954 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.771954 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:19.771954 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.771954 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.771954 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:19.771954 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.771954 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:19.774056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.774597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:19.782057 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:19.782705 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:19.783378 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:19.783692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.784525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:19.784525 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.784525 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:19.784525 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.784525 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.784525 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:19.784525 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.784525 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:19.786254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:19.787081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:19.787081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:19.787081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.787081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:19.787081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.787081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.787081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:19.787081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.787081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:19.787848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.788782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.789917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:19.789917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.789917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:19.883806 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:19.884146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:19.884626 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:19.885079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.885842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:19.885842 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.885842 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:19.885842 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.885842 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.885842 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:19.885842 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.885842 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:19.887039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:19.887972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:19.887972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.887972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:19.887972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.887972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.887972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:19.887972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.887972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:19.888364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.889127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:19.890149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.890149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:19.979091 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:19.979689 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:19.980230 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:19.980654 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:19.980900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.981811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:19.981811 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.981811 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:19.981811 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.981811 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.981811 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:19.981811 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.981811 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.983631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:19.984640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:24.596617 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:24.597780 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:24.633680 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:24.634018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.634767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:24.634767 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.634767 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:24.634767 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.634767 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.634767 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:24.634767 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.634767 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.635557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.636332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:24.770395 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:24.771886 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:24.772240 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:24.772473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.773494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:39:24.773494 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.773494 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:24.773494 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.773494 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.773494 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:24.773494 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.773494 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:24.775535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.776255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:24.782455 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:24.783070 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:24.783696 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:24.784005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.784833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:24.784833 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.784833 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:24.784833 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.784833 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.784833 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:24.784833 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.784833 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:24.786146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:24.786651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:24.786651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:24.786651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.786651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:24.786651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.786651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.786651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:24.786651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.786651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:24.788092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.788565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.789242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:24.789242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.789242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:24.884467 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:24.884989 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:24.885642 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:24.886002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.886857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:24.886857 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.886857 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:24.886857 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.886857 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.886857 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:24.886857 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.886857 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:24.888106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:24.888875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:24.888875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.888875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:24.888875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.888875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.888875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:24.888875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.888875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:24.889848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.890660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:24.891497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.891497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:24.979776 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:24.980390 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:24.981097 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:24.981556 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:24.981777 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.982729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:24.982729 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.982729 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:24.982729 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.982729 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.982729 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:24.982729 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.982729 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.984571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:24.985343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:29.597639 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:29.599295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:29.634364 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:29.634685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.635587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:29.635587 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.635587 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:29.635587 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.635587 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.635587 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:29.635587 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.635587 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.636556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.636984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:29.771739 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:29.773588 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:29.774036 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:29.774349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.775511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:39:29.775511 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.775511 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:29.775511 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.775511 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.775511 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:29.775511 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.775511 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:29.777697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.778536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:29.782907 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:29.783603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:29.784403 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:29.784756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.785674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:29.785674 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.785674 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:29.785674 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.785674 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.785674 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:29.785674 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.785674 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:29.787110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:29.787964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:29.787964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:29.787964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.787964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:29.787964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.787964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.787964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:29.787964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.787964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:29.788946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.789787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.790869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:29.790869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.790869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:29.885286 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:29.886134 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:29.886336 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:29.886717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.887717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:29.887717 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.887717 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:29.887717 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.887717 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.887717 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:29.887717 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.887717 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:29.888686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:29.889153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:29.889153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.889153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:29.889153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.889153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.889153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:29.889153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.889153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:29.889711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.890219 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:29.891004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.891004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:29.980147 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:29.980926 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:29.981464 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:29.981811 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:29.982038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.982914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:29.982914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.982914 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:29.982914 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.982914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.982914 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:29.982914 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.982914 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.984827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:29.985521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:34.598425 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:34.600050 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:34.609397 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=239 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:39:34.632465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:34.632949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.634132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:34.634132 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.634132 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:34.634132 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.634132 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.634132 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:34.634132 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.634132 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.635152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.636177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:34.772732 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:34.774509 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:34.774985 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:34.775310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.776616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:39:34.776616 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.776616 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:34.776616 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.776616 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.776616 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:34.776616 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.776616 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:34.778164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.778949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:34.783371 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:34.784059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:34.784734 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:34.785096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.786025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:34.786025 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.786025 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:34.786025 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.786025 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.786025 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:34.786025 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.786025 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:34.787521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:34.788459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:34.788459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:34.788459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.788459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:34.788459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.788459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.788459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:34.788459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.788459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:34.788918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.789636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.790452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:34.790452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.790452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:34.886010 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:34.886301 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:34.887440 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:34.887902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.889028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:34.889028 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.889028 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:34.889028 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.889028 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.889028 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:34.889028 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.889028 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:34.890442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:34.891170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:34.891170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.891170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:34.891170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.891170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.891170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:34.891170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.891170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:34.892164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.892955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:34.893819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.893819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:34.980757 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:34.981480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:34.982397 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:34.982876 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:34.983504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.984481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:34.984481 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.984481 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:34.984481 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.984481 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.984481 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:34.984481 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.984481 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.986958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:34.987848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:39.599021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:39.600452 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:39.632138 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:39.632359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.632857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:39.632857 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.632857 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:39.632857 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.632857 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.632857 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:39.632857 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.632857 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.633868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.634303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:39.773347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:39.774033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:39.774157 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:39.774284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.774895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:39:39.774895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.774895 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:39.774895 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.774895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.774895 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:39.774895 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.774895 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:39.775559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.775940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:39.784144 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:39.784841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:39.785370 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:39.785966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.786124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:39.786124 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.786124 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:39.786124 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.786124 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.786124 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:39.786124 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.786124 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:39.787473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:39.787957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:39.787957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:39.787957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.787957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:39.787957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.787957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.787957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:39.787957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.787957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:39.789157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.789672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.790278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:39.790278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.790278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:39.886570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:39.888288 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:39.888288 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:39.888288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:39.888487 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.888487 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:39.888487 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.888487 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.888487 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:39.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:39.889629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:39.890093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:39.890093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.890093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:39.890093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.890093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.890093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:39.890093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.890093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:39.891182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.891724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:39.892346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.892346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:39.981348 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:39.981794 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:39.982236 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:39.982561 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:39.982783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.983619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:39:39.983619 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.983619 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:39.983619 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.983619 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.983619 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:39.983619 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.983619 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.986037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:39.986861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:44.600133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:44.600735 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:44.630077 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:44.630266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.630699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:44.630699 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.630699 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:44.630699 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.630699 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.630699 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:44.630699 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.630699 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.631510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.632828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:44.774114 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:44.775457 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:44.775824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:44.776150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.777536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:39:44.777536 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.777536 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:44.777536 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.777536 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.777536 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:44.777536 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.777536 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:44.779254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.780171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:44.784823 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:44.785422 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:44.785812 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:44.786013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.786525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:44.786525 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.786525 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:44.786525 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.786525 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.786525 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:44.786525 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.786525 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:44.787450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:44.787874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:44.787874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:44.787874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.787874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:44.787874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.787874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.787874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:44.787874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.787874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:44.788752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.789194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.789685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:44.789685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.789685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:44.887268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:44.887548 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:44.888037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:44.888473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.889174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:44.889174 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.889174 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:44.889174 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.889174 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.889174 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:44.889174 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.889174 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:44.890411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:44.890753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:44.890753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.890753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:44.890753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.890753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.890753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:44.890753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.890753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:44.892098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.892441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:44.892798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.892798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:44.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:44.982589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:44.983185 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:44.983504 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:44.983665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.984371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:39:44.984371 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.984371 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:44.984371 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.984371 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.984371 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:44.984371 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.984371 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.985879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:44.986244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:49.630625 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:49.630625 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:49.656574 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:49.656914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.657769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:49.657769 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.657769 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:49.657769 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.657769 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.657769 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:49.657769 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.657769 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.658630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.659595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:49.774502 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:49.775609 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:49.775998 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:49.776395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.777592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:39:49.777592 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.777592 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:49.777592 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.777592 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.777592 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:49.777592 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.777592 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:49.779409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.780297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:49.785329 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:49.785709 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:49.786321 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:49.786396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.787021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:49.787021 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.787021 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:49.787021 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.787021 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.787021 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:49.787021 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.787021 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:49.788116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:49.788581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:49.788581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:49.788581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.788581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:49.788581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.788581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.788581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:49.788581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.788581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:49.789541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.789994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.790521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:49.790521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.790521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:49.887924 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:49.888377 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:49.889153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:49.889704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.890663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:49.890663 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.890663 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:49.890663 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.890663 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.890663 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:49.890663 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.890663 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:49.891927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:49.892702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:49.892702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.892702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:49.892702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.892702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.892702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:49.892702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.892702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:49.893544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.894323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:49.895287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.895287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:49.982424 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:49.982939 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:49.983563 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:49.983931 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:49.984158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.985036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:39:49.985036 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.985036 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:49.985036 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.985036 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.985036 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:49.985036 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.985036 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.986699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:49.987572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:54.600852 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:54.602513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:54.635513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:54.635803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.636404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:54.636404 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.636404 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:54.636404 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.636404 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.636404 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:54.636404 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.636404 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.637252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.638285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:54.775410 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:54.776886 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:54.777274 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:54.777516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.778612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:39:54.778612 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.778612 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:54.778612 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.778612 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.778612 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:54.778612 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.778612 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:54.780435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.780762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:54.785744 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:54.786448 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:54.786981 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:54.787353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.788115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:54.788115 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.788115 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:54.788115 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.788115 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.788115 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:54.788115 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.788115 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:54.789357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:54.789704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:54.789704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:54.789704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.789704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:54.789704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.789704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.789704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:54.789704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.789704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:54.790985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.791413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.791785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:54.791785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.791785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:54.888588 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:54.889001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:54.889709 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:54.890153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.890986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:54.890986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.890986 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:54.890986 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.890986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.890986 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:54.890986 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.890986 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:54.892124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:54.892874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:54.892874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.892874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:54.892874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.892874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.892874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:54.892874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.892874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:54.893704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.894545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:54.895592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.895592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:54.983209 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:54.983686 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:54.984194 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:54.984639 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:54.984854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.985660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:39:54.985660 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.985660 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:54.985660 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.985660 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.985660 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:54.985660 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.985660 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.987246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:54.988027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:59.601475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:59.602433 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:59.627631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:59.628068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.628973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:39:59.628973 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.628973 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:59.628973 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.628973 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.628973 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:59.628973 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.628973 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.629867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.630818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:59.776428 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:59.777769 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:59.778057 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:59.778329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:39:59.779198 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.779198 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:59.779198 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.779198 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.779198 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:59.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.779198 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:59.781072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.782027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:59.786317 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:59.786912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:39:59.787507 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:59.787507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.788184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:39:59.788184 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.788184 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:59.788184 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.788184 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.788184 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:59.788184 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.788184 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:59.789552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:59.790327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:59.790327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:59.790327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.790327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:59.790327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.790327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.790327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:59.790327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.790327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:39:59.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.792051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.792897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:59.792897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.792897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:59.889126 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:59.889579 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:59.890181 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:59.890670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.891649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:39:59.891649 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.891649 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:59.891649 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.891649 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.891649 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:59.891649 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.891649 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:39:59.893048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:59.893866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:39:59.893866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.893866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:39:59.893866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.893866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.893866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:39:59.893866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.893866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:39:59.895112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.896029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:59.897155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.897155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:39:59.984017 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:39:59.984697 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:39:59.985337 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:59.985763 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:39:59.986005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.986875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:39:59.986875 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.986875 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:39:59.986875 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.986875 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.986875 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:39:59.986875 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.986875 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.988580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:39:59.989419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:04.602952 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:04.603947 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:04.633411 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:04.633411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.634420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:04.634420 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.634420 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:04.634420 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.634420 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.634420 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:04.634420 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.634420 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.636804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.637542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:04.777330 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:04.778747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:04.778893 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:04.779118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.780017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:04.780017 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.780017 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:04.780017 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.780017 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.780017 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:04.780017 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.780017 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:04.781683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.782463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:04.787119 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:04.787606 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:04.787996 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:04.788290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.788867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:04.788867 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.788867 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:04.788867 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.788867 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.788867 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:04.788867 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.788867 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:04.790133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:04.791028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:04.791028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:04.791028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.791028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:04.791028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.791028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.791028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:04.791028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.791028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:04.791714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.792606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.793689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:04.793689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.793689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:04.889546 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:04.890621 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:04.891409 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:04.891834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.892734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:04.892734 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.892734 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:04.892734 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.892734 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.892734 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:04.892734 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.892734 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:04.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:04.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:04.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:04.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:04.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:04.895903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.896483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:04.897101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.897101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:04.984588 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:04.985754 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:04.986436 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:04.986794 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:04.987022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.988017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:40:04.988017 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.988017 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:04.988017 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.988017 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.988017 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:04.988017 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.988017 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.990145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:04.991019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:09.603017 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:09.604536 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:09.645202 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:09.645632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.646591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:09.646591 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.646591 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:09.646591 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.646591 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.646591 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:09.646591 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.646591 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.647571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.648131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:09.778068 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:09.778931 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:09.779284 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:09.779535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.780567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:09.780567 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.780567 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:09.780567 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.780567 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.780567 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:09.780567 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.780567 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:09.785970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.786746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:09.787522 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:09.788319 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:09.788875 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:09.789075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.789878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:09.789878 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.789878 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:09.789878 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.789878 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.789878 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:09.789878 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.789878 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:09.791243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:09.792381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:09.792381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:09.792381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.792381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:09.792381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.792381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.792381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:09.792381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.792381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:09.793304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.793878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.794458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:09.794458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.794458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:09.890378 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:09.890645 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:09.891366 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:09.891755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.892593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:09.892593 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.892593 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:09.892593 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.892593 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.892593 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:09.892593 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.892593 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:09.893969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:09.894830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:09.894830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.894830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:09.894830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.894830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.894830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:09.894830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.894830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:09.895523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.896335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:09.897151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.897151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:09.985207 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:09.985827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:09.986318 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:09.986663 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:09.986914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.987660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:40:09.987660 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.987660 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:09.987660 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.987660 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.987660 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:09.987660 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.987660 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.989240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:09.990047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:14.603657 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:14.604903 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:14.638569 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:14.638917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.639530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:14.639530 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.639530 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:14.639530 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.639530 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.639530 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:14.639530 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.639530 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.640710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.641896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:14.778498 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:14.780044 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:14.780418 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:14.780666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.781917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:14.781917 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.781917 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:14.781917 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.781917 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.781917 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:14.781917 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.781917 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:14.784620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.785590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:14.787983 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:14.788460 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:14.789253 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:14.789253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.789967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:14.789967 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.789967 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:14.789967 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.789967 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.789967 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:14.789967 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.789967 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:14.791163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:14.792113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:14.792113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:14.792113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.792113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:14.792113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.792113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.792113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:14.792113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.792113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:14.792725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.794324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.795509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:14.795509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.795509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:14.890953 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:14.891376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:14.891965 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:14.892341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.893128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:14.893128 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.893128 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:14.893128 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.893128 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.893128 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:14.893128 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.893128 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:14.894592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:14.895657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:14.895657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.895657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:14.895657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.895657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.895657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:14.895657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.895657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:14.896299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.897891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:14.898696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.898696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:14.985767 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:14.986579 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:14.987115 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:14.987482 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:14.987706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.988554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:40:14.988554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.988554 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:14.988554 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.988554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.988554 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:14.988554 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.988554 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.990364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:14.991430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:19.604433 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:19.610703 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:19.649352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:19.649952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.650601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:19.650601 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.650601 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:19.650601 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.650601 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.650601 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:19.650601 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.650601 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.652783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:19.779328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:19.780328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:19.780659 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:19.780877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.781868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:19.781868 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.781868 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:19.781868 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.781868 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.781868 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:19.781868 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.781868 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:19.783600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.784439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:19.788454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:19.789749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:19.789937 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:19.790410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.791241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:19.791241 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.791241 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:19.791241 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.791241 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.791241 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:19.791241 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.791241 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:19.792318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:19.793437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:19.794557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.795284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:19.891757 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:19.892250 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:19.892957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:19.893389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.894169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:19.894169 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.894169 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:19.894169 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.894169 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.894169 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:19.894169 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.894169 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:19.895595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:19.896501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:19.896501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.896501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:19.896501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.896501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.896501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:19.896501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.896501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:19.897200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.898066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:19.898851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.898851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:19.986330 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:19.987150 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:19.987706 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:19.988083 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:19.988348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.989164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:40:19.989164 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.989164 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:19.989164 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.989164 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.989164 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:19.989164 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.989164 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.991245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:19.992029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:24.607284 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:24.609648 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:24.644413 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:24.644730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.645358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:24.645358 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.645358 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:24.645358 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.645358 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.645358 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:24.645358 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.645358 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.646418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.647360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:24.779843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:24.781170 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:24.781553 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:24.781786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.782945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:24.782945 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.782945 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:24.782945 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.782945 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.782945 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:24.782945 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.782945 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:24.784864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.785697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:24.789090 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:24.789743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:24.790390 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:24.790682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.791454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:24.791454 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.791454 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:24.791454 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.791454 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.791454 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:24.791454 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.791454 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:24.792904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:24.793850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:24.793850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:24.793850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.793850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:24.793850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.793850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.793850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:24.793850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.793850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:24.794898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.795627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.796261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:24.796261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.796261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:24.892136 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:24.892801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:24.893532 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:24.893745 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.894527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:24.894527 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.894527 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:24.894527 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.894527 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.894527 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:24.894527 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.894527 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:24.896026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:24.896909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:24.896909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.896909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:24.896909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.896909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.896909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:24.896909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.896909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:24.897432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.898265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:24.899137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.899137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:24.986901 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:24.987460 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:24.987898 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:24.988356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:24.988653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.989471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:40:24.989471 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.989471 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:24.989471 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.989471 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.989471 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:24.989471 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.989471 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.991161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:24.992143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:29.608063 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:29.609461 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:29.644342 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:29.644917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.645925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:29.645925 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.645925 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:29.645925 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.645925 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.645925 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:29.645925 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.645925 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.647105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.647728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:29.780553 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:29.782008 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:29.782400 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:29.782656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.783576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:29.783576 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.783576 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:29.783576 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.783576 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.783576 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:29.783576 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.783576 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:29.784705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.785037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:29.789575 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:29.790259 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:29.791197 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:29.791197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.792328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:29.792328 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.792328 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:29.792328 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.792328 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.792328 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:29.792328 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.792328 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:29.793700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:29.794356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:29.794356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:29.794356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.794356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:29.794356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.794356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.794356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:29.794356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.794356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:29.795494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.795949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.796660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:29.796660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.796660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:29.892894 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:29.893349 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:29.894047 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:29.894486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.895514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:29.895514 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.895514 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:29.895514 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.895514 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.895514 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:29.895514 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.895514 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:29.896390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:29.896856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:29.896856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.896856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:29.896856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.896856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.896856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:29.896856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.896856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:29.897544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.898078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:29.898563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.898563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:29.987471 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:29.988099 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:29.989238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:29.989825 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:29.989919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.990772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:40:29.990772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.990772 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:29.990772 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.990772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.990772 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:29.990772 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.990772 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.994072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:29.994641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:34.608738 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:34.610188 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:34.616703 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=238 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:40:34.642426 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:34.642894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.643457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:34.643457 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.643457 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:34.643457 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.643457 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.643457 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:34.643457 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.643457 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.644346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.645077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:34.781353 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:34.782559 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:34.782870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:34.783106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.784036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:34.784036 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.784036 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:34.784036 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.784036 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.784036 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:34.784036 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.784036 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:34.785237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.785931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:34.790085 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:34.790945 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:34.791369 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:34.791651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.792315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:34.792315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.792315 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:34.792315 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.792315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.792315 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:34.792315 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.792315 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:34.793534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:34.794119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:34.794119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:34.794119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.794119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:34.794119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.794119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.794119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:34.794119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.794119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:34.794500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.794922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.795365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:34.795365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.795365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:34.893492 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:34.893784 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:34.894441 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:34.895027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.895606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:34.895606 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.895606 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:34.895606 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.895606 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.895606 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:34.895606 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.895606 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:34.896239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.896952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:34.897688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:34.898535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:34.898535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:34.898535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.898535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:34.898535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.898535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.898535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:34.898535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.898535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:34.988138 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:34.988877 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:34.989627 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:34.989884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:34.990137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.990935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:40:34.990935 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.990935 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:34.990935 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.990935 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.990935 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:34.990935 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.990935 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.992614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:34.993462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:39.609315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:39.611023 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:39.644248 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:39.644670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.645465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:39.645465 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.645465 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:39.645465 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.645465 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.645465 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:39.645465 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.645465 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.646586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.647246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:39.782020 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:39.783571 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:39.783957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:39.784228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.785315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:39.785315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.785315 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:39.785315 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.785315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.785315 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:39.785315 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.785315 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:39.787384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.787864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:39.790770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:39.791288 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:39.791827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:39.792334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.793358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:39.793358 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.793358 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:39.793358 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.793358 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.793358 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:39.793358 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.793358 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:39.795624 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:39.796846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:39.796846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:39.796846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.796846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:39.796846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.796846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.796846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:39.796846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.796846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:39.797592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.798559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.801288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:39.801288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.801288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:39.894154 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:39.894540 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:39.895178 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:39.895637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.896419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:39.896419 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.896419 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:39.896419 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.896419 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.896419 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:39.896419 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.896419 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:39.897633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:39.898426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:39.898426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.898426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:39.898426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.898426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.898426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:39.898426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.898426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:39.899483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.900468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:39.901343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.901343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:39.988679 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:39.989407 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:39.990151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:39.990586 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:39.990896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.991840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:40:39.991840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.991840 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:39.991840 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.991840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.991840 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:39.991840 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.991840 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.993718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:39.994620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:44.609981 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:44.611400 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:44.640539 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:44.641084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.642022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:44.642022 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.642022 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:44.642022 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.642022 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.642022 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:44.642022 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.642022 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.642908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.643847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:44.782687 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:44.784012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:44.784386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:44.784627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.785738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:44.785738 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.785738 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:44.785738 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.785738 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.785738 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:44.785738 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.785738 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:44.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.788607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:44.791351 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:44.792003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:44.792588 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:44.792897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.793749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:44.793749 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.793749 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:44.793749 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.793749 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.793749 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:44.793749 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.793749 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:44.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:44.795758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:44.795758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:44.795758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.795758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:44.795758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.795758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.795758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:44.795758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.795758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:44.796701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.797442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.798128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:44.798128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.798128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:44.895172 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.899677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.900008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.900351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:44.900668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:44.901038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.901038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:44.901038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.901038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.901038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:44.901038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.901038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:44.989368 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:44.989945 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:44.990463 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:44.990814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:44.991072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.991849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:40:44.991849 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.991849 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:44.991849 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.991849 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.991849 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:44.991849 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.991849 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.993477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:44.994551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:49.610697 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:49.612088 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:49.647552 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:49.647730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.648630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:49.648630 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.648630 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:49.648630 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.648630 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.648630 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:49.648630 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.648630 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.649727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.650623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:49.783394 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:49.784574 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:49.784904 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:49.785147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.786081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:49.786081 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.786081 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:49.786081 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.786081 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.786081 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:49.786081 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.786081 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:49.787922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.788568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:49.791814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:49.792311 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:49.792787 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:49.793075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.793795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:49.793795 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.793795 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:49.793795 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.793795 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.793795 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:49.793795 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.793795 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:49.794663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:49.795117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:49.795117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:49.795117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.795117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:49.795117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.795117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.795117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:49.795117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.795117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:49.795660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.796104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.796613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:49.796613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.796613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:49.895945 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:49.896202 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:49.896890 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:49.897411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.898124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:49.898124 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.898124 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:49.898124 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.898124 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.898124 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:49.898124 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.898124 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:49.899511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:49.899979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:49.899979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.899979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:49.899979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.899979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.899979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:49.899979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.899979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:49.901393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.901881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:49.902421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.902421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:49.990021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:49.990550 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:49.991072 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:49.991514 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:49.992306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.992538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:40:49.992538 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.992538 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:49.992538 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.992538 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.992538 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:49.992538 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.992538 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.994127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:49.994917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:54.611424 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:54.612842 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:54.648007 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:54.648703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.650157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:54.650157 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.650157 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:54.650157 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.650157 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.650157 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:54.650157 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.650157 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.651589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.652640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:54.784049 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:54.785579 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:54.786035 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:54.786337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.787865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:54.787865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.787865 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:54.787865 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.787865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.787865 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:54.787865 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.787865 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:54.789864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.790704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:54.792193 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:54.792989 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:54.793728 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:54.794088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.795236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:54.795236 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.795236 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:54.795236 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.795236 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.795236 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:54.795236 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.795236 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:54.796654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:54.797184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:54.797184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:54.797184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.797184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:54.797184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.797184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.797184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:54.797184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.797184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:54.797786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.798302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.798877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:54.798877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.798877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:54.896887 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:54.897620 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:54.898088 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:54.898755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.899591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:54.899591 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.899591 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:54.899591 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.899591 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.899591 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:54.899591 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.899591 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:54.901150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:54.901933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:54.901933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.901933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:54.901933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.901933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.901933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:54.901933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.901933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:54.902929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.903801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:54.905677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.905677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:54.990632 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:54.991059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:54.991918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:54.992449 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:54.992694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.993644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Traceback (most recent call last):
Mar 23 22:40:54.993644 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.993644 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:54.993644 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.993644 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.993644 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:54.993644 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.993644 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-04a733e6-3f15-4827-984e-92ef7d304c85 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.995317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:54.996162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:59.612356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:59.614272 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:59.648111 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:59.648791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.649822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:40:59.649822 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:59.649822 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:59.649822 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:59.649822 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:59.649822 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:59.649822 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.649822 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.650966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.652210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:59.784954 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:59.786501 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:59.786912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:59.787181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.788454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:40:59.788454 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:59.788454 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:59.788454 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:59.788454 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:59.788454 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:59.788454 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.788454 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:40:59.789899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.790434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:59.792985 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:59.793690 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:40:59.794337 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:59.794649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.795480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:40:59.795480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:59.795480 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:59.795480 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:59.795480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:59.795480 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:59.795480 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.795480 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:59.796861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:59.797342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:59.797342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:59.797342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:59.797342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:59.797342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:59.797342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:59.797342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:59.797342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.797342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:40:59.799673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:59.800018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:59.800447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:59.800447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.800447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:59.897567 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:59.898363 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:59.899082 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:59.899553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.900411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:59.900411 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:59.900411 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:59.900411 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:59.900411 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:59.900411 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:59.900411 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.900411 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:40:59.901982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:59.902758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:40:59.902758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:59.902758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:40:59.902758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:40:59.902758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:59.902758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:40:59.902758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.902758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:40:59.903695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:59.904580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:40:59.905419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.905419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:40:59.994659 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:40:59.995205 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:40:59.996475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:59.997011 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:40:59.997788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.999191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:40:59.999191 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:40:59.999191 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:40:59.999191 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:40:59.999191 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:40:59.999191 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:40:59.999191 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:40:59.999191 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:00.001286 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:00.001850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:04.613328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:04.615547 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:04.650301 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:04.650619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.650912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:04.650912 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:04.650912 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:04.650912 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:04.650912 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:04.650912 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:04.650912 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.650912 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.651754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.652400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:04.785787 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:04.787051 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:04.787431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:04.788111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.788526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:04.788526 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:04.788526 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:04.788526 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:04.788526 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:04.788526 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:04.788526 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.788526 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:04.790489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.791289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:04.793519 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:04.793920 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:04.794519 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:04.794796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.795593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:41:04.795593 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:04.795593 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:04.795593 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:04.795593 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:04.795593 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:04.795593 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.795593 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:04.796902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:04.797530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:04.797530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:04.797530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:04.797530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:04.797530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:04.797530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:04.797530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:04.797530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.797530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:04.798571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:04.799043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:04.799563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:04.799563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.799563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:04.898364 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:04.898810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:04.899467 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:04.899764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.900440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:04.900440 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:04.900440 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:04.900440 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:04.900440 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:04.900440 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:04.900440 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.900440 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:04.901906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:04.902826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:04.902826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:04.902826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:04.902826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:04.902826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:04.902826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:04.902826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.902826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:04.903871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:04.904781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:04.905766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.905766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:04.995561 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:04.997197 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:04.998229 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:04.998229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.999077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:04.999077 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:04.999077 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:04.999077 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:04.999077 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:04.999077 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:04.999077 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:04.999077 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:05.000261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:05.000881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:09.614484 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:09.616382 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:09.647957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:09.648277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.649027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:09.649027 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:09.649027 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:09.649027 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:09.649027 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:09.649027 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:09.649027 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.649027 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.650205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.650837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:09.786985 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:09.788402 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:09.788791 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:09.789046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.790143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:09.790143 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:09.790143 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:09.790143 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:09.790143 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:09.790143 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:09.790143 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.790143 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:09.792042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.793058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:09.794109 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:09.794614 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:09.795361 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:09.795727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.796753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:41:09.796753 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:09.796753 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:09.796753 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:09.796753 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:09.796753 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:09.796753 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.796753 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:09.798369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:09.798905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:09.798905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:09.798905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:09.798905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:09.798905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:09.798905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:09.798905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:09.798905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.798905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:09.800102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:09.800668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:09.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:09.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:09.899389 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:09.899991 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:09.900814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:09.901338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.902253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:09.902253 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:09.902253 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:09.902253 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:09.902253 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:09.902253 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:09.902253 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.902253 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:09.903156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.904180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:09.904919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:09.905806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:09.905806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:09.905806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:09.905806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:09.905806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:09.905806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:09.905806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:09.905806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.905806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:09.996273 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:09.997120 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:09.997726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:09.997998 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:09.998152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.998806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:09.998806 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:09.998806 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:09.998806 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:09.998806 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:09.998806 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:09.998806 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:09.998806 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:10.000709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:10.001710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:14.615251 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:14.617012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:14.648109 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:14.648583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:14.649443 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:14.649443 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:14.649443 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:14.649443 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:14.649443 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:14.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.649443 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.650595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.651087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:14.788174 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:14.789581 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:14.789858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:14.790084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.791080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:14.791080 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:14.791080 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:14.791080 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:14.791080 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:14.791080 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:14.791080 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.791080 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:14.792900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.793722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:14.794794 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:14.795133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:14.795671 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:14.796020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.796839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:41:14.796839 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:14.796839 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:14.796839 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:14.796839 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:14.796839 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:14.796839 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.796839 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:14.798249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:14.798869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:14.798869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:14.798869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:14.798869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:14.798869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:14.798869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:14.798869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:14.798869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.798869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:14.799731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:14.800318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:14.800833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:14.800833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.800833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:14.900107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:14.900475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:14.900775 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:14.901030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.901647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:14.901647 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:14.901647 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:14.901647 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:14.901647 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:14.901647 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:14.901647 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.901647 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:14.902856 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:14.903890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:14.903890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:14.903890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:14.903890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:14.903890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:14.903890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:14.903890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.903890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:14.904360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:14.904943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:14.905510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.905510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:14.996765 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:14.997365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:14.998031 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:14.998515 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:14.998766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.999608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:14.999608 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:14.999608 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:14.999608 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:14.999608 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:14.999608 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:14.999608 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:14.999608 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:15.001165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:15.002081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:19.616105 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:19.617821 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:19.646690 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:19.646911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.647451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:19.647451 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:19.647451 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:19.647451 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:19.647451 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:19.647451 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:19.647451 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.647451 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.648338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.649172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:19.788760 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:19.790103 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:19.790185 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:19.790269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.791048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:19.791048 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:19.791048 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:19.791048 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:19.791048 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:19.791048 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:19.791048 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.791048 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:19.792902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.793455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:19.795151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:19.795684 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:19.796033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:19.796184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.797543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:41:19.797543 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:19.797543 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:19.797543 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:19.797543 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:19.797543 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:19.797543 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.797543 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:19.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:19.800019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:19.800019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:19.800019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:19.800019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:19.800019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:19.800019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:19.800019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:19.800019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.800019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:19.800979 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:19.801488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:19.802035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:19.802035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.802035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:19.900512 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:19.900752 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:19.901104 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:19.901310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.901892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:19.901892 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:19.901892 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:19.901892 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:19.901892 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:19.901892 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:19.901892 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.901892 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:19.903169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:19.903907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:19.903907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:19.903907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:19.903907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:19.903907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:19.903907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:19.903907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.903907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:19.904910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:19.905480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:19.906019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.906019 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:19.997317 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:19.998237 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:19.998570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:19.998770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:19.998948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.999515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:19.999515 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:19.999515 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:19.999515 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:19.999515 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:19.999515 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:19.999515 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:19.999515 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:20.000699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:20.001209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:24.616685 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:24.618638 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:24.653256 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:24.653666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.654686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:24.654686 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:24.654686 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:24.654686 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:24.654686 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:24.654686 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:24.654686 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.654686 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.655878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.656880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:24.789793 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:24.791025 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:24.791332 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:24.791514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.792719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:24.792719 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:24.792719 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:24.792719 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:24.792719 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:24.792719 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:24.792719 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.792719 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:24.794731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.795651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:24.797383 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:24.797383 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:24.797383 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:24.797383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.797851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:41:24.797851 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:24.797851 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:24.797851 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:24.797851 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:24.797851 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:24.797851 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.797851 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:24.799088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:24.800058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:24.800058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:24.800058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:24.800058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:24.800058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:24.800058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:24.800058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:24.800058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.800058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:24.800718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:24.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:24.802395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:24.802395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.802395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:24.901130 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:24.901770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:24.902489 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:24.902721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.903342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:24.903342 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:24.903342 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:24.903342 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:24.903342 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:24.903342 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:24.903342 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.903342 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:24.904767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:24.905636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:24.905636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:24.905636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:24.905636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:24.905636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:24.905636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:24.905636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.905636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:24.906700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:24.907591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:24.908376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:24.908376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:24.998239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:24.999083 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:24.999762 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:25.000272 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:25.000554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:25.001561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:25.001561 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:25.001561 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:25.001561 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:25.001561 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:25.001561 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:25.001561 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:25.001561 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:25.003830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:25.004651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:29.617527 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:29.619110 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:29.652295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:29.653097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.653461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:29.653461 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:29.653461 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:29.653461 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:29.653461 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:29.653461 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:29.653461 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.653461 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.654629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.655740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:29.790365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:29.791798 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:29.792001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:29.792197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.793197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:29.793197 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:29.793197 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:29.793197 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:29.793197 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:29.793197 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:29.793197 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.793197 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:29.795268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.796065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:29.796999 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:29.797246 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:29.797835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:29.798160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.798999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:41:29.798999 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:29.798999 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:29.798999 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:29.798999 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:29.798999 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:29.798999 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.798999 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:29.800583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:29.801360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:29.801360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:29.801360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:29.801360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:29.801360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:29.801360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:29.801360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:29.801360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.801360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:29.802623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:29.803103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:29.803588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:29.803588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.803588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:29.901953 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:29.902156 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:29.903019 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:29.903391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.904144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:29.904144 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:29.904144 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:29.904144 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:29.904144 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:29.904144 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:29.904144 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.904144 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:29.905241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:29.906054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:29.906054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:29.906054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:29.906054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:29.906054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:29.906054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:29.906054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.906054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:29.906689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:29.907792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:29.908789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:29.908789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:29.999095 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:29.999732 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:30.000416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:30.000816 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:30.001066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:30.001960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:30.001960 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:30.001960 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:30.001960 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:30.001960 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:30.001960 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:30.001960 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:30.001960 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:30.003896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:30.004974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:34.618174 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:34.619870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:34.626164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:41:34.652385 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:34.652641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.653191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:34.653191 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:34.653191 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:34.653191 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:34.653191 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:34.653191 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:34.653191 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.653191 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.653981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.654760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:34.791064 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:34.792239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:34.792493 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:34.792665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.793345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:34.793345 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:34.793345 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:34.793345 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:34.793345 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:34.793345 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:34.793345 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.793345 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:34.795278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.796548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:34.797483 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:34.797483 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:34.797483 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:34.797639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.798355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:41:34.798355 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:34.798355 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:34.798355 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:34.798355 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:34.798355 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:34.798355 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.798355 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:34.799758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:34.800412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:34.800412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:34.800412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:34.800412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:34.800412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:34.800412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:34.800412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:34.800412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.800412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:34.801322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:34.801929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:34.802472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:34.802472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.802472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:34.902458 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:34.902671 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:34.903088 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:34.903150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.903509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:34.903509 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:34.903509 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:34.903509 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:34.903509 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:34.903509 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:34.903509 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.903509 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:34.904315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:34.904828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:34.904828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:34.904828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:34.904828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:34.904828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:34.904828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:34.904828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.904828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:34.905376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:34.905838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:34.906508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:34.906508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:34.999986 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:35.000906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:35.001599 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:35.001888 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:35.002077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:35.002888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:41:35.002888 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:35.002888 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:35.002888 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:35.002888 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:35.002888 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:35.002888 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:35.002888 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:35.005012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:35.005916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:39.619370 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:39.621733 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:39.659344 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:39.659617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.660228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:39.660228 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:39.660228 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:39.660228 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:39.660228 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:39.660228 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:39.660228 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.660228 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.661087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.662036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:39.792122 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:39.793769 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:39.794110 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:39.794384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.795518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:39.795518 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:39.795518 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:39.795518 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:39.795518 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:39.795518 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:39.795518 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.795518 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:39.797407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.798340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:39.799454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:39.799454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:39.799454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:39.799746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.800372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:39.800372 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:39.800372 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:39.800372 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:39.800372 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:39.800372 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:39.800372 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.800372 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:39.801106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:39.802298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:39.803501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.804477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:39.903131 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:39.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:39.904348 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:39.904771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.905798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:39.905798 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:39.905798 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:39.905798 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:39.905798 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:39.905798 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:39.905798 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.905798 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:39.908078 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.908970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:39.909711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:39.910663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:39.910663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:39.910663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:39.910663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:39.910663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:39.910663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:39.910663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:39.910663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:39.910663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:40.000711 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:40.001686 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:40.002786 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:40.003175 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:40.003485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:40.004419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:41:40.004419 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:40.004419 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:40.004419 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:40.004419 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:40.004419 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:40.004419 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:40.004419 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:40.005875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:40.006413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:44.620169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:44.622152 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:44.654003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:44.654262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.654775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:44.654775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:44.654775 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:44.654775 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:44.654775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:44.654775 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:44.654775 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.654775 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.655942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.656932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:44.793198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:44.794575 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:44.794774 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:44.794873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.795633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:44.795633 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:44.795633 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:44.795633 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:44.795633 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:44.795633 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:44.795633 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.795633 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:44.797420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.798267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:44.799146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:44.799146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:44.799332 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:44.799398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.800027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:41:44.800027 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:44.800027 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:44.800027 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:44.800027 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:44.800027 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:44.800027 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.800027 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:44.801503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:44.801953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:44.801953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:44.801953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:44.801953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:44.801953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:44.801953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:44.801953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:44.801953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.801953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:44.803065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:44.803531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:44.804021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:44.804021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.804021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:44.903887 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:44.904994 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:44.905232 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:44.905433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.906342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:44.906342 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:44.906342 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:44.906342 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:44.906342 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:44.906342 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:44.906342 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.906342 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:44.906895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:44.907623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:44.907623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:44.907623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:44.907623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:44.907623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:44.907623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:44.907623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.907623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:44.907974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:44.909058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:44.909831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:44.909831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:45.001591 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:45.002346 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:45.002833 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:45.003092 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:45.003301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:45.003777 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:41:45.003777 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:45.003777 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:45.003777 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:45.003777 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:45.003777 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:45.003777 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:45.003777 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:45.004990 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:45.005559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:49.621107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:49.622371 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:49.657084 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:49.657469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.658159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:49.658159 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:49.658159 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:49.658159 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:49.658159 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:49.658159 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:49.658159 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.658159 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.658952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.659559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:49.793965 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:49.795378 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:49.795622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:49.795831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.796678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:49.796678 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:49.796678 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:49.796678 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:49.796678 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:49.796678 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:49.796678 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.796678 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:49.798672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.799421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:49.800203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:49.800203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:49.800203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:49.800203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.800823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:41:49.800823 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:49.800823 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:49.800823 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:49.800823 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:49.800823 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:49.800823 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.800823 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:49.801593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:49.802375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:49.802375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:49.802375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:49.802375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:49.802375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:49.802375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:49.802375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:49.802375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.802375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:49.802891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:49.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:49.804011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:49.804011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.804011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:49.905088 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:49.905598 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:49.906255 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:49.906792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.907672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:49.907672 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:49.907672 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:49.907672 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:49.907672 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:49.907672 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:49.907672 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.907672 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:49.908965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:49.909675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:49.909675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:49.909675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:49.909675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:49.909675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:49.909675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:49.909675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.909675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:49.910592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:49.911342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:49.912113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:49.912113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:50.002281 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:50.002968 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:50.003461 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:50.003883 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:50.004120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:50.005022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:50.005022 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:50.005022 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:50.005022 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:50.005022 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:50.005022 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:50.005022 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:50.005022 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:50.007044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:50.007863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:54.621806 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:54.623500 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:54.661484 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:54.662028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.663585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:54.663585 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:54.663585 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:54.663585 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:54.663585 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:54.663585 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:54.663585 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.663585 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.664720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.665887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:54.794813 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:54.796235 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:54.796680 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:54.796950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.798059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:54.798059 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:54.798059 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:54.798059 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:54.798059 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:54.798059 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:54.798059 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.798059 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:54.800031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.800622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:54.801103 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:54.801103 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:54.801275 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:54.801596 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.802370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:41:54.802370 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:54.802370 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:54.802370 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:54.802370 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:54.802370 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:54.802370 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.802370 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:54.803169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:54.803799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:54.803799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:54.803799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:54.803799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:54.803799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:54.803799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:54.803799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:54.803799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.803799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:54.804174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:54.804874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:54.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:54.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:54.905964 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:54.906431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:54.907198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:54.907746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.908759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:54.908759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:54.908759 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:54.908759 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:54.908759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:54.908759 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:54.908759 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.908759 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:54.909591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.910561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:54.911276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:54.912099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:54.912099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:54.912099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:54.912099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:54.912099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:54.912099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:54.912099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:54.912099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:54.912099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:55.003132 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:55.003746 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:55.004121 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:55.004363 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:55.004485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:55.005225 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:41:55.005225 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:55.005225 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:55.005225 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:55.005225 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:55.005225 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:55.005225 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:55.005225 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:55.006149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:55.006544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:59.622420 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:59.624595 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:59.658761 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:59.659340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.660423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:41:59.660423 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:59.660423 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:59.660423 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:59.660423 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:59.660423 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:59.660423 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.660423 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.661460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.662199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:59.795398 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:59.796764 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:59.797160 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:59.797427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.798375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:41:59.798375 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:59.798375 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:59.798375 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:59.798375 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:59.798375 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:59.798375 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.798375 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:41:59.799682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.800800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:59.801679 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:59.801679 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:41:59.801932 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:59.802275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.803171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:41:59.803171 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:59.803171 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:59.803171 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:59.803171 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:59.803171 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:59.803171 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.803171 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:59.804551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:59.805175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:59.805175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:59.805175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:59.805175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:59.805175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:59.805175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:59.805175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:59.805175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.805175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:41:59.806137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:59.806723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:59.807247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:59.807247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.807247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:41:59.906597 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:41:59.907075 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:41:59.907686 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:41:59.908109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.908966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:41:59.908966 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:59.908966 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:41:59.908966 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:41:59.908966 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:59.908966 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:41:59.908966 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.908966 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:41:59.910345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:59.911145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:41:59.911145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:59.911145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:41:59.911145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:41:59.911145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:59.911145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:41:59.911145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.911145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:41:59.911942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:41:59.912779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:41:59.913641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:41:59.913641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:00.003697 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:00.004116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:00.004622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:00.004972 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:00.005203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:00.006109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:42:00.006109 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:00.006109 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:00.006109 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:00.006109 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:00.006109 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:00.006109 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:00.006109 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:00.007582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:00.008485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:04.623326 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:04.624987 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:04.656809 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:04.657368 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.658801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:04.658801 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:04.658801 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:04.658801 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:04.658801 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:04.658801 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:04.658801 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.658801 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.660004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.661107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:04.796263 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:04.797665 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:04.798019 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:04.798234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.799204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:04.799204 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:04.799204 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:04.799204 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:04.799204 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:04.799204 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:04.799204 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.799204 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:04.800986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.801870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:04.802799 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:04.802799 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:04.802799 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:04.802799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.803203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:42:04.803203 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:04.803203 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:04.803203 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:04.803203 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:04.803203 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:04.803203 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.803203 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:04.804423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:04.805313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:04.805313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:04.805313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:04.805313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:04.805313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:04.805313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:04.805313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:04.805313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.805313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:04.806004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:04.806873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:04.807957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:04.807957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.807957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:04.907121 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:04.907514 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:04.908486 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:04.908941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:42:04.909832 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:04.909832 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:04.909832 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:04.909832 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:04.909832 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:04.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.909832 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:04.911075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:04.911932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:04.911932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:04.911932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:04.911932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:04.911932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:04.911932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:04.911932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.911932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:04.912626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:04.913513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:04.914395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:04.914395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:05.004258 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:05.004846 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:05.005551 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:05.006010 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:05.006344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:05.007199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:42:05.007199 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:05.007199 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:05.007199 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:05.007199 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:05.007199 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:05.007199 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:05.007199 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:05.008598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:05.009742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:09.624083 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:09.625756 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:09.657406 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:09.657819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.658761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:09.658761 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:09.658761 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:09.658761 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:09.658761 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:09.658761 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:09.658761 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.658761 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.659913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.660759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:09.796963 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:09.799053 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:09.799386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:09.799622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.800661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:09.800661 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:09.800661 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:09.800661 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:09.800661 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:09.800661 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:09.800661 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.800661 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:09.802389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.803256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:09.804189 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:09.804189 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:09.804189 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:09.804189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:42:09.804443 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:09.804443 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:09.804443 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:09.804443 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:09.804443 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:09.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:09.804843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:09.805566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:09.806256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.807108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:09.907884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:09.908444 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:09.909159 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:09.909685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.910846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:42:09.910846 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:09.910846 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:09.910846 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:09.910846 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:09.910846 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:09.910846 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.910846 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:09.912120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:09.912688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:09.912688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:09.912688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:09.912688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:09.912688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:09.912688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:09.912688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:09.912688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.912688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:09.913792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:09.914388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:09.915050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:09.915050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:10.004863 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:10.005506 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:10.006203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:10.006748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:10.006901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:10.007775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:42:10.007775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:10.007775 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:10.007775 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:10.007775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:10.007775 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:10.007775 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:10.007775 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:10.009980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:10.011018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:14.624770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:14.626419 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:14.660822 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:14.661337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.662134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:14.662134 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:14.662134 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:14.662134 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:14.662134 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:14.662134 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:14.662134 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.662134 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.663312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.664134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:14.797857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:14.799042 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:14.799419 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:14.799665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.800547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:14.800547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:14.800547 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:14.800547 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:14.800547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:14.800547 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:14.800547 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.800547 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:14.802356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.803188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:14.804024 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:14.804024 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:14.804574 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:14.804849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.805746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:42:14.805746 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:14.805746 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:14.805746 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:14.805746 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:14.805746 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:14.805746 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.805746 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:14.807238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:14.807927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:14.807927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:14.807927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:14.807927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:14.807927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:14.807927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:14.807927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:14.807927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.807927 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:14.808397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:14.809148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:14.810014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:14.810014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.810014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:14.908480 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:14.908961 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:14.909554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:14.909914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.910885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:42:14.910885 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:14.910885 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:14.910885 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:14.910885 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:14.910885 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:14.910885 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.910885 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:14.912022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:14.912706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:14.912706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:14.912706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:14.912706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:14.912706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:14.912706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:14.912706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:14.912706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.912706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:14.913059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:14.913701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:14.914541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:14.914541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:15.005697 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:15.006549 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:15.007111 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:15.007393 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:15.007549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:15.008287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Traceback (most recent call last):
Mar 23 22:42:15.008287 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:15.008287 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:15.008287 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:15.008287 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:15.008287 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:15.008287 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:15.008287 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-5700d6d4-2097-4607-8396-649c0f0ee21f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:15.009998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:15.010953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:19.643069 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:19.645016 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:19.674347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:19.674765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.675691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:19.675691 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:19.675691 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:19.675691 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:19.675691 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:19.675691 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:19.675691 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.675691 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.676571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.677673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:19.798547 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:19.799701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:19.799997 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:19.800236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.801341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:19.801341 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:19.801341 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:19.801341 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:19.801341 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:19.801341 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:19.801341 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.801341 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:19.803142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.804002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:19.804932 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:19.804932 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:19.804932 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:19.804932 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.805732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:42:19.805732 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:19.805732 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:19.805732 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:19.805732 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:19.805732 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:19.805732 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.805732 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:19.807060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:19.807897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:19.807897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:19.807897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:19.807897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:19.807897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:19.807897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:19.807897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:19.807897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.807897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:19.808676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:19.809510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:19.810458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:19.810458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.810458 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:19.908978 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:19.909176 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:19.909926 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:19.910306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.911025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:42:19.911025 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:19.911025 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:19.911025 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:19.911025 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:19.911025 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:19.911025 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.911025 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:19.911850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:19.912647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:19.912647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:19.912647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:19.912647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:19.912647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:19.912647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:19.912647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:19.912647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.912647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:19.913133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:19.913988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:19.914951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:19.914951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:20.006363 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:20.007029 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:20.007567 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:20.007915 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:20.008147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:20.009057 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:42:20.009057 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:20.009057 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:20.009057 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:20.009057 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:20.009057 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:20.009057 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:20.009057 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:20.010840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:20.011426 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:24.643458 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:24.645976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:24.689820 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:24.690250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.691301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:24.691301 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:24.691301 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:24.691301 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:24.691301 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:24.691301 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:24.691301 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.691301 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.693329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.693748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:24.799919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:24.800468 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:24.800730 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:24.800907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.804159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:24.804159 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:24.804159 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:24.804159 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:24.804159 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:24.804159 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:24.804159 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.804159 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:24.805235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.805743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:24.806366 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:24.806825 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:24.807497 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:24.807710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.808300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:42:24.808300 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:24.808300 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:24.808300 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:24.808300 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:24.808300 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:24.808300 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.808300 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:24.813269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:24.813808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:24.814313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.814871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:24.909407 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:24.910105 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:24.910745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:24.910971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.911554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:42:24.911554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:24.911554 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:24.911554 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:24.911554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:24.911554 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:24.911554 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.911554 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:24.912580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:24.913151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:24.913151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:24.913151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:24.913151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:24.913151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:24.913151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:24.913151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:24.913151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.913151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:24.913658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:24.914326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:24.915080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:24.915080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:25.006988 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:25.008043 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:25.008703 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:25.009104 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:25.009359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:25.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:25.013128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:25.013621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:25.013621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:25.013621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:25.013621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:25.013621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:25.013621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:29.644171 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:29.646950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:29.673432 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:29.674189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.675251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:29.675251 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:29.675251 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:29.675251 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:29.675251 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:29.675251 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:29.675251 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.675251 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.676079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.676963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:29.799757 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:29.801073 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:29.801368 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:29.801720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.802450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:29.802450 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:29.802450 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:29.802450 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:29.802450 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:29.802450 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:29.802450 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.802450 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:29.804443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.805404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:29.806707 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:29.807199 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:29.807736 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:29.808012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.809071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:42:29.809071 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:29.809071 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:29.809071 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:29.809071 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:29.809071 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:29.809071 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.809071 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:29.810012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:29.810469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:29.810469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:29.810469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:29.810469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:29.810469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:29.810469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:29.810469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:29.810469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.810469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:29.811231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:29.812297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:29.812735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:29.812735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.812735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:29.910246 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:29.910828 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:29.912335 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:29.912335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.912459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:42:29.912459 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:29.912459 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:29.912459 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:29.912459 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:29.912459 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:29.912459 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.912459 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:29.913254 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:29.913729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:29.913729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:29.913729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:29.913729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:29.913729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:29.913729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:29.913729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:29.913729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.913729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:29.914272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:29.915391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:29.916386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:29.916386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:30.007902 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:30.008763 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:30.009107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:30.009313 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:30.009492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:30.012501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:30.013099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:30.013725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:30.013725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:30.013725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:30.013725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:34.644853 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:34.646886 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:34.648995 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=239 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:42:34.679185 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:34.679185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.679185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:34.679185 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:34.679185 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:34.679185 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:34.679185 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:34.679185 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:34.679185 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.679185 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.680010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.680622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:34.800490 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:34.801339 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:34.801460 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:34.801724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.802056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:34.802056 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:34.802056 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:34.802056 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:34.802056 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:34.802056 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:34.802056 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.802056 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:34.802829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.803633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:34.807366 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:34.808023 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:34.808490 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:34.808490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.808896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:42:34.808896 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:34.808896 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:34.808896 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:34.808896 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:34.808896 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:34.808896 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.808896 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:34.809790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:34.810357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:34.810357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:34.810357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:34.810357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:34.810357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:34.810357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:34.810357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:34.810357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.810357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:34.811780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:34.812510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:34.813312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:34.813312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.813312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:34.910921 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:34.912231 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:34.912807 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:34.913043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.916660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.917441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:34.917825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:34.918170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:34.918641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:34.918641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:35.008930 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:35.009556 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:35.010125 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:35.010357 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:35.010572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:35.011004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:42:35.011004 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:35.011004 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:35.011004 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:35.011004 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:35.011004 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:35.011004 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:35.011004 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:35.012737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:35.013735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:39.358465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:39.368662 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:389}}
Mar 23 22:42:39.368801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=104232) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:370}}
Mar 23 22:42:39.368995 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Fixing resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:42:39.409710 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, resource={'id': '06cffee1-b8de-433e-a67a-ec0e950c9238', 'name': 'private', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['130adef4-306b-4397-a096-91ee5094d0e2', 'fb1912da-e258-4d41-8cd4-7896f0de6385'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:17:25Z', 'updated_at': '2026-03-23T22:17:28Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 36968}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.410241 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.410904 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c4286b-d53b-4217-a9f3-05177afa9297, external_ids_update=None, columns={'tag': 36968}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.411116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.412443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:42:39.412443 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.412443 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:39.412443 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.412443 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.412443 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:39.412443 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.412443 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:39.413839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Failed to fix resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.413839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:42:39.413839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:42:39.413839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:42:39.413839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:42:39.413839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:42:39.413839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:42:39.413839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:39.413839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.413839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.413839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.414588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.415190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:42:39.415190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.415190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:42:39.415190 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Fixing resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:42:39.625207 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, resource={'id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'name': 'public', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'a53179c9-417a-4560-9bc7-e3f64551b15b'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-23T22:17:31Z', 'updated_at': '2026-03-23T22:17:40Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.625631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.626510 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a8bd993e-d23e-4e73-8066-766307763bdb, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.626780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.627590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:42:39.627590 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.627590 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:39.627590 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.627590 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.627590 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:39.627590 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.627590 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:39.628899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Failed to fix resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.628899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:42:39.628899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:42:39.628899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:42:39.628899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:42:39.628899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:42:39.628899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:42:39.628899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:39.628899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.628899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.628899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.629150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.629429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:42:39.629429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.629429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:42:39.629429 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Fixing resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:42:39.645748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:39.649256 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:39.675690 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, resource={'id': '3a042d54-7a79-465e-bd57-47cd6833a4d0', 'name': 'admin_net', 'tenant_id': 'b4ffb2af173e4612a30978ef510f4f44', 'project_id': 'b4ffb2af173e4612a30978ef510f4f44', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['935d20c1-7d6b-489a-8358-b5fb94945b7e'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:20:57Z', 'updated_at': '2026-03-23T22:20:58Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 47875}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.676550 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.676783 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c442d0-efff-4c7e-bcb6-377b4da39c34, external_ids_update=None, columns={'tag': 47875}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.677105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.678018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:42:39.678018 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.678018 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:39.678018 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.678018 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.678018 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:39.678018 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.678018 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:39.679655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Failed to fix resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.679655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:42:39.679655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:42:39.679655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:42:39.679655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:42:39.679655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:42:39.679655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:42:39.679655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:39.679655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.679655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.679655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.679981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:42:39.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:42:39.680337 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Fixing resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:42:39.733064 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.733623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.736901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:39.736901 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.736901 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:39.736901 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.736901 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.736901 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:39.736901 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.736901 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.738366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.739525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:39.740438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, resource={'id': 'c9e6c30a-b30e-473b-b06a-50be75c90b36', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['421749b7-bb0d-4854-b3f4-f0414f4c37fb'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:21:34Z', 'updated_at': '2026-03-23T22:21:35Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 48210}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.740438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.740625 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13535482-a705-4453-af87-e8a7d8c10c82, external_ids_update=None, columns={'tag': 48210}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.740969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.741403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:42:39.741403 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.741403 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:39.741403 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.741403 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.741403 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:39.741403 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.741403 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:39.742517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Failed to fix resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.742517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:42:39.742517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:42:39.742517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:42:39.742517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:42:39.742517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:42:39.742517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:42:39.742517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:39.742517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.742517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.742517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.742825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.743170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:42:39.743170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.743170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:42:39.743170 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Fixing resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:42:39.801047 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:39.805761 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.806055 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.806944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.807999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:39.807999 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.807999 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:39.807999 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.807999 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.807999 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:39.807999 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.807999 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:39.811001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.811900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:39.812728 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:39.815155 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:39.815879 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.816840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.816840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:39.816840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.816840 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:39.816840 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.816840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.816840 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:39.816840 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.816840 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:39.818203 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:39.818770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.818770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:39.818770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.818770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:39.818770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.818770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.818770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:39.818770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.818770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:39.819332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.819771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.820275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:39.820275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.820275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:39.820623 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, resource={'id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'enable_snat': True}, 'gw_port_id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-t62km', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'ip_address': '172.24.5.146', 'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a'}, {'ip_address': '2001:db8::396', 'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 3, 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d'}, resource_type=routers, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.821323 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.821605 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.821809 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.821994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.823035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:42:39.823035 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.823035 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:39.823035 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.823035 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.823035 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:39.823035 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.823035 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:39.823276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-96b50231-d848-437c-9393-29d871719f74 None None] Unable to update router 154c08d6-78c5-49fd-a785-0df3f4277a23. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.824852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Failed to fix resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.824852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:42:39.824852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:42:39.824852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:42:39.824852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:42:39.824852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:42:39.824852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 23 22:42:39.824852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 23 22:42:39.824852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 23 22:42:39.824852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 23 22:42:39.824852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:42:39.825214 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:42:39.825599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.825599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:42:39.825599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.825599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:42:39.825599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.825599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.825599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:42:39.825599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.825599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:42:39.825599 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Fixing resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:42:39.914337 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:39.915173 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:39.916181 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.917520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.918649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:42:39.918649 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.918649 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:39.918649 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.918649 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.918649 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:39.918649 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.918649 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:39.920520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:39.920989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.920989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:39.920989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.920989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:39.920989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.920989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.920989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:39.920989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.920989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:39.922589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.923438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:39.924304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.924304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:39.929752 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, resource={'id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'name': '', 'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'project_id': '', 'mac_address': 'fa:16:3e:aa:0e:57', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160382', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.930140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a a53179c9-417a-4560-9bc7-e3f64551b15b', 'neutron:network_name': 'neutron-bb9d5c47-4694-42c2-a018-18e3c9058915', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-154c08d6-78c5-49fd-a785-0df3f4277a23'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::396/64', '172.24.5.146/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.930548 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=348aba7c-88a8-4a19-9486-df3f7edb4007, lrouter_port=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:39.930822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.931510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:42:39.931510 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.931510 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:39.931510 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.931510 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.931510 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:39.931510 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.931510 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:39.932705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Failed to fix resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.932705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:42:39.932705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:42:39.932705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:42:39.932705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:42:39.932705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:42:39.932705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 23 22:42:39.932705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:39.932705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.932705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.932705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:42:39.933030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:39.933416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:42:39.933416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:39.933416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:42:39.933416 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96b50231-d848-437c-9393-29d871719f74 None None] Maintenance task: Synchronization completed (took 0.56 seconds)
Mar 23 22:42:40.009495 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:40.009821 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:40.010714 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:40.011150 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:40.011518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:40.012310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:42:40.012310 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:40.012310 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:40.012310 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:40.012310 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:40.012310 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:40.012310 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:40.012310 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:40.013074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:40.013879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:44.646989 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:44.648847 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:44.675468 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:44.675834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.676681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:44.676681 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:44.676681 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:44.676681 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:44.676681 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:44.676681 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:44.676681 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.676681 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.677484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.678323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:44.803813 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:44.804716 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:44.804841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:44.804938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.805460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:42:44.805460 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:44.805460 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:44.805460 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:44.805460 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:44.805460 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:44.805460 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.805460 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:44.806244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.807052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:44.815328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:44.815835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:44.816164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:44.816309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.816745 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:44.816745 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:44.816745 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:44.816745 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:44.816745 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:44.816745 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:44.816745 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.816745 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:44.817718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:44.818773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:44.818773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:44.818773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:44.818773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:44.818773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:44.818773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:44.818773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:44.818773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.818773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:44.819380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:44.820146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:44.821033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:44.821033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.821033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:44.915286 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:44.915835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:44.916635 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:44.917065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.917984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:42:44.917984 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:44.917984 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:44.917984 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:44.917984 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:44.917984 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:44.917984 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.917984 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:44.919440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:44.919923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:44.919923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:44.919923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:44.919923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:44.919923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:44.919923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:44.919923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:44.919923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.919923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:44.920880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:44.921390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:44.921866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:44.921866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:45.010285 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:45.016249 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:45.017037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:45.017584 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:45.017944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:45.019560 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:42:45.019560 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:45.019560 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:45.019560 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:45.019560 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:45.019560 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:45.019560 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:45.019560 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:45.021157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:45.021872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:49.647673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:49.648891 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:49.681198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:49.681552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.682237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:49.682237 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:49.682237 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:49.682237 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:49.682237 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:49.682237 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:49.682237 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.682237 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.683115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.683910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:49.804494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:49.806437 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:49.806842 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:49.806980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.808153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:42:49.808153 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:49.808153 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:49.808153 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:49.808153 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:49.808153 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:49.808153 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.808153 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:49.809963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.810899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:49.815887 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:49.816548 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:49.817033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:49.817403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.818204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:49.818204 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:49.818204 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:49.818204 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:49.818204 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:49.818204 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:49.818204 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.818204 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:49.819253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:49.820018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:49.820018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:49.820018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:49.820018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:49.820018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:49.820018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:49.820018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:49.820018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.820018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:49.820498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:49.821312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:49.822190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:49.822190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.822190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:49.915796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:49.916393 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:49.917112 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:49.917614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.918501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:42:49.918501 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:49.918501 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:49.918501 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:49.918501 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:49.918501 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:49.918501 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.918501 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:49.919829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:49.920564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:49.920564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:49.920564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:49.920564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:49.920564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:49.920564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:49.920564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:49.920564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.920564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:49.921605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:49.922545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:49.923421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:49.923421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:50.016209 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:50.017109 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:50.018107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:50.018602 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:50.019416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:50.019886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:42:50.019886 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:50.019886 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:50.019886 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:50.019886 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:50.019886 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:50.019886 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:50.019886 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:50.021389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:50.022140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:54.648354 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:54.649932 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:54.684523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:54.684915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.685779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:54.685779 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:54.685779 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:54.685779 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:54.685779 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:54.685779 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:54.685779 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.685779 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.686645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.687056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:54.805276 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:54.806748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:54.807153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:54.807323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.807964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:42:54.807964 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:54.807964 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:54.807964 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:54.807964 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:54.807964 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:54.807964 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.807964 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:54.809169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.810010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:54.816431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:54.816952 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:54.817369 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:54.817557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.818480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:54.818480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:54.818480 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:54.818480 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:54.818480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:54.818480 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:54.818480 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.818480 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:54.819375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:54.819961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:54.819961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:54.819961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:54.819961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:54.819961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:54.819961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:54.819961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:54.819961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.819961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:54.820419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:54.821117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:54.822018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:54.822018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.822018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:54.916477 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:54.917000 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:54.917708 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:54.918080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.919027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:42:54.919027 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:54.919027 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:54.919027 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:54.919027 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:54.919027 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:54.919027 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.919027 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:54.920398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:54.921058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:54.921058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:54.921058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:54.921058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:54.921058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:54.921058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:54.921058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:54.921058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.921058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:54.922016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:54.922853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:54.923749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:54.923749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:55.016758 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:55.017111 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:55.017655 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:55.018016 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:55.018250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:55.019051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:42:55.019051 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:55.019051 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:55.019051 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:55.019051 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:55.019051 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:55.019051 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:55.019051 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:55.020632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:55.021469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:59.649130 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:59.650817 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:59.685622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:59.686050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.686846 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:42:59.686846 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:59.686846 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:59.686846 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:59.686846 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:59.686846 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:59.686846 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.686846 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.687922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.688884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:59.805998 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:59.807117 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:59.807503 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:59.807741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.808811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:42:59.808811 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:59.808811 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:59.808811 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:59.808811 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:59.808811 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:59.808811 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.808811 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:42:59.810681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.811582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:59.816969 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:59.817503 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:42:59.818086 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:59.818432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.819456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:42:59.819456 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:59.819456 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:59.819456 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:59.819456 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:59.819456 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:59.819456 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.819456 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:59.820751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:59.821343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:59.821343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:59.821343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:59.821343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:59.821343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:59.821343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:59.821343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:59.821343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.821343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:42:59.822972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:59.823477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:59.824005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:59.824005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.824005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:42:59.917328 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:42:59.918083 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:42:59.919193 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:42:59.919193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.920119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:42:59.920119 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:59.920119 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:42:59.920119 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:42:59.920119 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:59.920119 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:42:59.920119 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.920119 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:42:59.921710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:42:59.922582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:59.922582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:42:59.922582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:59.922582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:42:59.922582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:42:59.922582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:59.922582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:42:59.922582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.922582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:42:59.923433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:42:59.924291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:42:59.925451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:42:59.925451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:00.017540 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:00.018137 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:00.018773 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:00.019255 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:00.019539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:00.020530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:00.020530 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:00.020530 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:00.020530 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:00.020530 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:00.020530 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:00.020530 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:00.020530 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:00.022006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:00.022872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:04.649838 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:04.651392 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:04.685308 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:04.685744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.686610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:04.686610 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:04.686610 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:04.686610 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:04.686610 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:04.686610 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:04.686610 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.686610 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.687772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.688852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:04.806881 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:04.808365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:04.808593 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:04.808793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.810036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:04.810036 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:04.810036 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:04.810036 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:04.810036 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:04.810036 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:04.810036 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.810036 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:04.811866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.812402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:04.817509 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:04.818167 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:04.818599 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:04.818971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.819727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:04.819727 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:04.819727 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:04.819727 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:04.819727 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:04.819727 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:04.819727 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.819727 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:04.820895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:04.822120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:04.822120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:04.822120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:04.822120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:04.822120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:04.822120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:04.822120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:04.822120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.822120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:04.822564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:04.823240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:04.823866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:04.823866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.823866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:04.918092 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:04.919085 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:04.919831 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:04.919831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.920595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:04.920595 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:04.920595 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:04.920595 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:04.920595 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:04.920595 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:04.920595 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.920595 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:04.922328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:04.922754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:04.922754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:04.922754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:04.922754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:04.922754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:04.922754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:04.922754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:04.922754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.922754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:04.923679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:04.924051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:04.924408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:04.924408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:05.018415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:05.018984 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:05.019661 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:05.020039 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:05.020321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:05.021156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:05.021156 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:05.021156 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:05.021156 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:05.021156 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:05.021156 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:05.021156 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:05.021156 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:05.022760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:05.023747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:09.650380 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:09.652016 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:09.685859 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:09.686310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.688018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:09.688018 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:09.688018 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:09.688018 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:09.688018 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:09.688018 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:09.688018 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.688018 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.688955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.690025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:09.807521 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:09.808867 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:09.809178 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:09.809493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.810532 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:09.810532 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:09.810532 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:09.810532 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:09.810532 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:09.810532 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:09.810532 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.810532 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:09.812689 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.813605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:09.818057 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:09.818607 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:09.819229 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:09.819585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.820367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:09.820367 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:09.820367 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:09.820367 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:09.820367 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:09.820367 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:09.820367 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.820367 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:09.821693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:09.822653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:09.822653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:09.822653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:09.822653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:09.822653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:09.822653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:09.822653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:09.822653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.822653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:09.823373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:09.823805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:09.824283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:09.824283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.824283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:09.918822 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:09.919454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:09.920134 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:09.920631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.921486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:09.921486 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:09.921486 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:09.921486 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:09.921486 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:09.921486 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:09.921486 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.921486 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:09.922901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:09.923773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:09.923773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:09.923773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:09.923773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:09.923773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:09.923773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:09.923773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:09.923773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.923773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:09.924195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:09.924987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:09.925876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:09.925876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:10.019191 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:10.019745 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:10.020270 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:10.020789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:10.021056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:10.022058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:10.022058 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:10.022058 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:10.022058 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:10.022058 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:10.022058 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:10.022058 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:10.022058 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:10.023948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:10.024952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:14.651089 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:14.652384 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:14.689378 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:14.689764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.690714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:14.690714 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:14.690714 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:14.690714 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:14.690714 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:14.690714 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:14.690714 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.690714 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.691589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.692538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:14.808296 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:14.809637 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:14.809966 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:14.810206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.811079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:14.811079 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:14.811079 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:14.811079 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:14.811079 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:14.811079 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:14.811079 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.811079 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:14.812388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.813402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:14.818549 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:14.819148 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:14.819796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:14.820084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.820920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:14.820920 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:14.820920 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:14.820920 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:14.820920 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:14.820920 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:14.820920 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.820920 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:14.822520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:14.822970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:14.822970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:14.822970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:14.822970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:14.822970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:14.822970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:14.822970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:14.822970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.822970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:14.824732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:14.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:14.825781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:14.825781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.825781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:14.919391 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:14.919824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:14.920465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:14.920860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.921968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:14.921968 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:14.921968 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:14.921968 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:14.921968 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:14.921968 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:14.921968 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.921968 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:14.922640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:14.923377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:14.924052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:14.924968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:15.019884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:15.020453 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:15.020954 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:15.021392 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:15.021667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:15.022481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:15.022481 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:15.022481 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:15.022481 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:15.022481 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:15.022481 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:15.022481 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:15.022481 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:15.032095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:15.033011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:19.651877 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:19.685522 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:19.718660 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:19.719257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.720168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:19.720168 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:19.720168 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:19.720168 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:19.720168 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:19.720168 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:19.720168 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.720168 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.721333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.721682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:19.808973 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:19.810153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:19.810554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:19.810836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.811809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:19.811809 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:19.811809 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:19.811809 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:19.811809 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:19.811809 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:19.811809 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.811809 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:19.812732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.813102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:19.818970 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:19.819525 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:19.819901 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:19.820098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.820718 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:19.820718 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:19.820718 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:19.820718 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:19.820718 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:19.820718 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:19.820718 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.820718 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:19.821902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:19.822252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:19.822252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:19.822252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:19.822252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:19.822252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:19.822252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:19.822252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:19.822252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.822252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:19.823129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:19.823463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:19.823792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:19.823792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.823792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:19.919979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:19.920473 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:19.921010 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:19.921538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.922489 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:19.922489 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:19.922489 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:19.922489 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:19.922489 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:19.922489 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:19.922489 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.922489 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:19.923795 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:19.924111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:19.924111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:19.924111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:19.924111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:19.924111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:19.924111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:19.924111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:19.924111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.924111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:19.925491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:19.925791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:19.926150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:19.926150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:20.020315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:20.021075 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:20.021815 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:20.022234 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:20.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:20.023112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:20.023112 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:20.023112 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:20.023112 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:20.023112 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:20.023112 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:20.023112 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:20.023112 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:20.024866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:20.026100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:24.684408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:24.685577 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:24.719626 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:24.719878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.720757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:24.720757 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:24.720757 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:24.720757 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:24.720757 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:24.720757 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:24.720757 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.720757 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.721866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.722432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:24.809898 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:24.811452 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:24.811861 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:24.812337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.813090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:24.813090 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:24.813090 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:24.813090 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:24.813090 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:24.813090 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:24.813090 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.813090 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:24.814973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.815496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:24.819610 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:24.820385 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:24.820906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:24.821234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.822011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:24.822011 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:24.822011 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:24.822011 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:24.822011 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:24.822011 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:24.822011 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.822011 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:24.823001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:24.823840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:24.823840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:24.823840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:24.823840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:24.823840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:24.823840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:24.823840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:24.823840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.823840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:24.824377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:24.825160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:24.826092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:24.826092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.826092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:24.920641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:24.920863 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:24.921231 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:24.921545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.922131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:24.922131 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:24.922131 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:24.922131 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:24.922131 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:24.922131 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:24.922131 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.922131 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:24.922944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:24.923675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:24.923675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:24.923675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:24.923675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:24.923675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:24.923675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:24.923675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:24.923675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.923675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:24.924138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:24.924925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:24.925793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:24.925793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:25.021071 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:25.021608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:25.022576 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:25.023054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:25.023345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:25.024173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:25.024173 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:25.024173 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:25.024173 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:25.024173 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:25.024173 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:25.024173 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:25.024173 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:25.025861 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:25.026913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:29.685146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:29.686774 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:29.718050 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:29.718345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.719242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:29.719242 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:29.719242 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:29.719242 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:29.719242 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:29.719242 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:29.719242 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.719242 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.720006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.720635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:29.810689 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:29.811991 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:29.812589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:29.813063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.813881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:29.813881 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:29.813881 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:29.813881 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:29.813881 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:29.813881 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:29.813881 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.813881 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:29.815695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.816522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:29.820374 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:29.821075 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:29.821322 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:29.821674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.822298 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:29.822298 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:29.822298 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:29.822298 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:29.822298 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:29.822298 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:29.822298 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.822298 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:29.823058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:29.823528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:29.823528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:29.823528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:29.823528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:29.823528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:29.823528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:29.823528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:29.823528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.823528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:29.824040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:29.824520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:29.825026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:29.825026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.825026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:29.921120 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:29.921545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:29.921793 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:29.921975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.922384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:29.922384 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:29.922384 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:29.922384 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:29.922384 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:29.922384 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:29.922384 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.922384 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:29.922951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:29.923654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:29.923654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:29.923654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:29.923654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:29.923654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:29.923654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:29.923654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:29.923654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.923654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:29.924039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:29.924883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:29.925691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:29.925691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:30.021716 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:30.022383 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:30.022942 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:30.023365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:30.023598 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:30.024407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:30.024407 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:30.024407 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:30.024407 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:30.024407 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:30.024407 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:30.024407 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:30.024407 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:30.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:30.026985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:34.685986 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:34.687755 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:34.689907 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=434 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:43:34.722708 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:34.723110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.724047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:34.724047 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:34.724047 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:34.724047 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:34.724047 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:34.724047 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:34.724047 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.724047 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.724984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.725566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:34.811387 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:34.812366 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:34.812684 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:34.813000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.813772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:34.813772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:34.813772 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:34.813772 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:34.813772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:34.813772 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:34.813772 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.813772 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:34.815252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.816072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:34.821048 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:34.821750 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:34.822371 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:34.822663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.823512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:34.823512 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:34.823512 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:34.823512 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:34.823512 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:34.823512 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:34.823512 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.823512 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:34.824775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:34.825641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:34.825641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:34.825641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:34.825641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:34.825641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:34.825641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:34.825641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:34.825641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.825641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:34.826322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:34.827149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:34.828080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:34.828080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.828080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:34.921686 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:34.922094 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:34.922678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:34.923020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.923803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:34.923803 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:34.923803 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:34.923803 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:34.923803 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:34.923803 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:34.923803 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.923803 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:34.924984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:34.925555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:34.925555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:34.925555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:34.925555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:34.925555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:34.925555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:34.925555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:34.925555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.925555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:34.926184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:34.926711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:34.927191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:34.927191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:35.022280 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:35.022898 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:35.023500 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:35.023911 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:35.024138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:35.024895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:35.024895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:35.024895 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:35.024895 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:35.024895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:35.024895 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:35.024895 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:35.024895 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:35.026715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:35.027494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:39.696198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:39.697714 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:39.727710 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:39.728011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.728702 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:39.728702 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:39.728702 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:39.728702 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:39.728702 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:39.728702 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:39.728702 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.728702 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.729944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.730959 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:39.812269 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:39.813462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:39.813693 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:39.813848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.814577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:39.814577 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:39.814577 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:39.814577 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:39.814577 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:39.814577 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:39.814577 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.814577 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:39.815905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.817311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:39.821605 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:39.822003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:39.822380 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:39.822573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.823444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:39.823444 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:39.823444 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:39.823444 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:39.823444 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:39.823444 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:39.823444 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.823444 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:39.824782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:39.825289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:39.825289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:39.825289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:39.825289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:39.825289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:39.825289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:39.825289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:39.825289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.825289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:39.826554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:39.827107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:39.827657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:39.827657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.827657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:39.922381 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:39.923068 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:39.924042 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:39.924545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.925462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:39.925462 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:39.925462 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:39.925462 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:39.925462 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:39.925462 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:39.925462 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.925462 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:39.926384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:39.927105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:39.927105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:39.927105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:39.927105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:39.927105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:39.927105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:39.927105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:39.927105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.927105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:39.927576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:39.928412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:39.929388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:39.929388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:40.023031 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:40.023847 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:40.024405 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:40.024870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:40.025187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:40.026121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:40.026121 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:40.026121 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:40.026121 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:40.026121 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:40.026121 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:40.026121 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:40.026121 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:40.027392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:40.028793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:44.696198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:44.697824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:44.732955 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:44.733343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.734050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:44.734050 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:44.734050 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:44.734050 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:44.734050 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:44.734050 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:44.734050 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.734050 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.734833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.735787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:44.813968 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:44.815380 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:44.815679 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:44.815918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.816950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:44.816950 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:44.816950 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:44.816950 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:44.816950 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:44.816950 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:44.816950 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.816950 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:44.819670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.820508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:44.822165 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:44.822727 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:44.823258 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:44.823498 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.824070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:44.824070 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:44.824070 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:44.824070 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:44.824070 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:44.824070 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:44.824070 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.824070 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:44.825402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:44.826206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:44.826206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:44.826206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:44.826206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:44.826206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:44.826206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:44.826206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:44.826206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.826206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:44.827055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:44.827554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:44.828034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:44.828034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.828034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:44.923153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:44.923770 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:44.924490 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:44.925034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.925738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:44.925738 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:44.925738 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:44.925738 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:44.925738 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:44.925738 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:44.925738 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.925738 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:44.926956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:44.927700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:44.927700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:44.927700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:44.927700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:44.927700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:44.927700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:44.927700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:44.927700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.927700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:44.928826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:44.930088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:44.930928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:44.930928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:45.023897 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:45.024664 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:45.025253 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:45.025462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:45.025568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:45.026162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:45.026162 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:45.026162 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:45.026162 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:45.026162 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:45.026162 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:45.026162 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:45.026162 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:45.028242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:45.029455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:49.697254 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:49.699006 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:49.730091 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:49.730315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.730845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:49.730845 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:49.730845 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:49.730845 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:49.730845 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:49.730845 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:49.730845 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.730845 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.731843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.732793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:49.814652 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:49.815743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:49.816023 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:49.816259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.817129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:49.817129 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:49.817129 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:49.817129 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:49.817129 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:49.817129 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:49.817129 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.817129 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:49.819095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.820309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:49.822645 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:49.823320 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:49.823880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:49.824122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.824817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:49.824817 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:49.824817 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:49.824817 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:49.824817 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:49.824817 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:49.824817 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.824817 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:49.825693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:49.826325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:49.826325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:49.826325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:49.826325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:49.826325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:49.826325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:49.826325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:49.826325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.826325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:49.826663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:49.827165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:49.827780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:49.827780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.827780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:49.923754 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:49.924247 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:49.924912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:49.925365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.926157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:49.926157 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:49.926157 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:49.926157 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:49.926157 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:49.926157 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:49.926157 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.926157 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:49.927440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:49.928144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:49.928144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:49.928144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:49.928144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:49.928144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:49.928144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:49.928144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:49.928144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.928144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:49.929153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:49.929957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:49.930879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:49.930879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:50.024538 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:50.024950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:50.025468 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:50.025811 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:50.026025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:50.027077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:50.027077 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:50.027077 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:50.027077 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:50.027077 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:50.027077 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:50.027077 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:50.027077 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:50.028804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:50.029739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:54.698071 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:54.700036 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:54.733926 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:54.734302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.735142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:54.735142 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:54.735142 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:54.735142 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:54.735142 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:54.735142 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:54.735142 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.735142 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.735992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.737005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:54.815395 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:54.816826 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:54.817194 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:54.817479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.818572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:54.818572 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:54.818572 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:54.818572 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:54.818572 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:54.818572 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:54.818572 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.818572 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:54.820580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.821308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:54.823238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:54.823881 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:54.824478 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:54.824760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.825570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:54.825570 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:54.825570 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:54.825570 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:54.825570 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:54.825570 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:54.825570 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.825570 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:54.826915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:54.827472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:54.827472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:54.827472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:54.827472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:54.827472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:54.827472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:54.827472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:54.827472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.827472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:54.828712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:54.829198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:54.829764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:54.829764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.829764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:54.924633 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:54.925119 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:54.925549 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:54.925777 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.926419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:54.926419 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:54.926419 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:54.926419 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:54.926419 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:54.926419 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:54.926419 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.926419 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:54.927727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:54.928316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:54.928316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:54.928316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:54.928316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:54.928316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:54.928316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:54.928316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:54.928316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.928316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:54.929405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:54.930741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:54.931277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:54.931277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:55.025322 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:55.025720 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:55.026254 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:55.026601 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:55.026825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:55.027568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:43:55.027568 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:55.027568 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:55.027568 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:55.027568 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:55.027568 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:55.027568 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:55.027568 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:55.028814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:55.029641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:59.698824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:59.700391 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:59.733604 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:59.734446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.735585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:43:59.735585 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:59.735585 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:59.735585 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:59.735585 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:59.735585 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:59.735585 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.735585 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.736858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.737823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:59.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:59.817654 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:59.818092 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:59.818404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.819361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:43:59.819361 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:59.819361 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:59.819361 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:59.819361 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:59.819361 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:59.819361 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.819361 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:43:59.821082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.822042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:59.823858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:59.824409 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:43:59.824806 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:59.825010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.825520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:43:59.825520 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:59.825520 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:59.825520 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:59.825520 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:59.825520 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:59.825520 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.825520 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:59.826496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:59.827248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:59.827248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:59.827248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:59.827248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:59.827248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:59.827248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:59.827248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:59.827248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.827248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:43:59.827681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:59.828403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:59.829161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:59.829161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.829161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:43:59.925320 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:43:59.925862 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:43:59.926343 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:43:59.926575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.927165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:43:59.927165 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:59.927165 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:43:59.927165 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:43:59.927165 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:59.927165 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:43:59.927165 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.927165 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:43:59.928579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:43:59.929561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:59.929561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:43:59.929561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:59.929561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:43:59.929561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:43:59.929561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:59.929561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:43:59.929561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.929561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:43:59.930311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:43:59.930800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:43:59.931287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:43:59.931287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:00.025956 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:00.026439 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:00.027025 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:00.027655 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:00.027893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:00.028939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:44:00.028939 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:00.028939 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:00.028939 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:00.028939 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:00.028939 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:00.028939 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:00.028939 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:00.031150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:00.031977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:04.699551 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:04.700600 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:04.736852 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:04.737083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.737625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:04.737625 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:04.737625 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:04.737625 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:04.737625 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:04.737625 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:04.737625 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.737625 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.738709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.739642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:04.816710 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:04.817977 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:04.818391 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:04.818714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.819695 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:04.819695 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:04.819695 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:04.819695 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:04.819695 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:04.819695 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:04.819695 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.819695 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:04.821440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.823105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:04.824365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:04.825003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:04.825575 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:04.825767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.826420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:04.826420 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:04.826420 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:04.826420 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:04.826420 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:04.826420 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:04.826420 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.826420 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:04.827945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:04.828451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:04.828451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:04.828451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:04.828451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:04.828451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:04.828451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:04.828451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:04.828451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.828451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:04.829773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:04.830284 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:04.830802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:04.830802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.830802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:04.925874 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:04.926509 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:04.927127 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:04.927568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.928407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:04.928407 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:04.928407 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:04.928407 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:04.928407 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:04.928407 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:04.928407 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.928407 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:04.929738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:04.930815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:04.930815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:04.930815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:04.930815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:04.930815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:04.930815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:04.930815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:04.930815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.930815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:04.931482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:04.932528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:04.933605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:04.933605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:05.026673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:05.027499 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:05.028120 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:05.028531 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:05.028758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:05.029665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:44:05.029665 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:05.029665 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:05.029665 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:05.029665 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:05.029665 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:05.029665 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:05.029665 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:05.031751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:05.032320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:09.700305 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:09.702105 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:09.732171 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:09.732508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.733116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:09.733116 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:09.733116 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:09.733116 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:09.733116 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:09.733116 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:09.733116 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.733116 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.734093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.734651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:09.817308 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:09.818737 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:09.818923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:09.819366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.820092 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:09.820092 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:09.820092 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:09.820092 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:09.820092 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:09.820092 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:09.820092 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.820092 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:09.822021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.822878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:09.824931 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:09.825250 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:09.825706 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:09.826378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.826573 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:09.826573 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:09.826573 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:09.826573 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:09.826573 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:09.826573 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:09.826573 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.826573 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:09.827460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:09.827954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:09.827954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:09.827954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:09.827954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:09.827954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:09.827954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:09.827954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:09.827954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.827954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:09.828756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:09.829454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:09.830124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:09.830124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.830124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:09.926820 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:09.927746 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:09.928321 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:09.929009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.929597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:09.929597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:09.929597 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:09.929597 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:09.929597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:09.929597 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:09.929597 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.929597 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:09.930982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:09.931667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:09.931667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:09.931667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:09.931667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:09.931667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:09.931667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:09.931667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:09.931667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.931667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:09.932686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:09.933402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:09.933890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:09.933890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:10.027457 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:10.027928 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:10.028477 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:10.028910 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:10.029118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:10.029818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:44:10.029818 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:10.029818 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:10.029818 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:10.029818 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:10.029818 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:10.029818 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:10.029818 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:10.031044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:10.031437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:14.701248 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:14.702362 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:14.735239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:14.735463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.736053 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:14.736053 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:14.736053 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:14.736053 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:14.736053 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:14.736053 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:14.736053 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.736053 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.737201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.738838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:14.818049 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:14.819018 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:14.819514 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:14.819753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.820817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:14.820817 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:14.820817 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:14.820817 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:14.820817 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:14.820817 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:14.820817 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.820817 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:14.822654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.823429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:14.825410 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:14.825734 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:14.826175 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:14.826450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.827292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:14.827292 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:14.827292 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:14.827292 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:14.827292 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:14.827292 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:14.827292 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.827292 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:14.828471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:14.828934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:14.828934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:14.828934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:14.828934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:14.828934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:14.828934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:14.828934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:14.828934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.828934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:14.830029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:14.830495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:14.830991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:14.830991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.830991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:14.927605 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:14.928256 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:14.928856 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:14.929249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.930074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:14.930074 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:14.930074 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:14.930074 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:14.930074 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:14.930074 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:14.930074 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.930074 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:14.931607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:14.932306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:14.932306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:14.932306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:14.932306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:14.932306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:14.932306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:14.932306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:14.932306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.932306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:14.933415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:14.934147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:14.935064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:14.935064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:15.028098 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:15.028719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:15.029248 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:15.029631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:15.029842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:15.030541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:44:15.030541 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:15.030541 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:15.030541 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:15.030541 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:15.030541 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:15.030541 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:15.030541 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:15.032155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:15.033032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:19.714991 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:19.715954 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:19.740102 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:19.740327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.740879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:19.740879 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:19.740879 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:19.740879 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:19.740879 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:19.740879 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:19.740879 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.740879 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.741863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.742554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:19.818569 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:19.819831 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:19.820039 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:19.820193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.821653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:19.821653 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:19.821653 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:19.821653 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:19.821653 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:19.821653 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:19.821653 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.821653 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:19.822867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.823369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:19.825739 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:19.826399 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:19.826911 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:19.827074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.830411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.831020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:19.831462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:19.831928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:19.832493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:19.832493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.832493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:19.928168 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:19.928772 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:19.929423 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:19.929705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.930857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:19.930857 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:19.930857 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:19.930857 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:19.930857 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:19.930857 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:19.930857 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.930857 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:19.932011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:19.932579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:19.932579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:19.932579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:19.932579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:19.932579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:19.932579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:19.932579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:19.932579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.932579 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:19.933819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:19.934488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:19.935306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:19.935306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:20.028769 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:20.029375 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:20.030041 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:20.030360 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:20.030526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:20.031141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:44:20.031141 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:20.031141 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:20.031141 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:20.031141 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:20.031141 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:20.031141 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:20.031141 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:20.032874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:20.033265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:24.714923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:24.716645 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:24.751612 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:24.751964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.752828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:24.752828 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:24.752828 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:24.752828 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:24.752828 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:24.752828 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:24.752828 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.752828 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.753852 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.754621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:24.819443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:24.820721 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:24.820950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:24.821182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.822058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:24.822058 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:24.822058 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:24.822058 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:24.822058 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:24.822058 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:24.822058 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.822058 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:24.823068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.823656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:24.826289 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:24.827070 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:24.827615 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:24.827800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.828510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:24.828510 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:24.828510 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:24.828510 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:24.828510 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:24.828510 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:24.828510 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.828510 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:24.829931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:24.830259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:24.830259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:24.830259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:24.830259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:24.830259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:24.830259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:24.830259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:24.830259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.830259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:24.833246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:24.834652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:24.835163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:24.835163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.835163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:24.929264 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:24.929264 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:24.929893 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:24.930301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.931065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:24.931065 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:24.931065 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:24.931065 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:24.931065 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:24.931065 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:24.931065 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.931065 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:24.931945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:24.932491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:24.932491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:24.932491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:24.932491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:24.932491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:24.932491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:24.932491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:24.932491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.932491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:24.933086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:24.933548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:24.933892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:24.933892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:25.029312 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:25.029816 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:25.030275 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:25.030552 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:25.030693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:25.031351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:44:25.031351 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:25.031351 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:25.031351 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:25.031351 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:25.031351 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:25.031351 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:25.031351 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:25.032633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:25.033287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:29.715558 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:29.716665 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:29.748497 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:29.748701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.749399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:29.749399 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:29.749399 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:29.749399 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:29.749399 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:29.749399 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:29.749399 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.749399 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.750265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.750952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:29.820584 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:29.823276 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:29.823276 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:29.823276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.823276 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:29.823276 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:29.823276 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:29.823276 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:29.823276 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:29.823276 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:29.823276 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.823276 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:29.824184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.825002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:29.826683 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:29.827040 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:29.827666 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:29.828051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.828608 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:29.828608 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:29.828608 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:29.828608 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:29.828608 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:29.828608 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:29.828608 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.828608 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:29.829667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:29.830159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:29.830159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:29.830159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:29.830159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:29.830159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:29.830159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:29.830159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:29.830159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.830159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:29.831562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:29.832013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:29.832549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:29.832549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.832549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:29.929296 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:29.929778 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:29.931427 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:29.931427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.931634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:29.931634 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:29.931634 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:29.931634 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:29.931634 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:29.931634 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:29.931634 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.931634 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:29.932524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:29.933409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:29.933409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:29.933409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:29.933409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:29.933409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:29.933409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:29.933409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:29.933409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.933409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:29.933790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:29.934536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:29.935399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:29.935399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:30.029783 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:30.030361 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:30.031157 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:30.032827 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:30.032827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:30.032967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:44:30.032967 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:30.032967 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:30.032967 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:30.032967 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:30.032967 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:30.032967 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:30.032967 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:30.033789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:30.034450 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:34.716472 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:34.717782 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:34.719268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:44:34.758427 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:34.758613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.759199 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:34.759199 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:34.759199 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:34.759199 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:34.759199 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:34.759199 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:34.759199 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.759199 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.760166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.760833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:34.821284 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:34.822716 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:34.823162 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:34.823545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.824629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:34.824629 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:34.824629 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:34.824629 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:34.824629 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:34.824629 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:34.824629 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.824629 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:34.826511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.827000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:34.827710 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:34.827710 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:34.828474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:34.828790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.829616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:34.829616 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:34.829616 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:34.829616 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:34.829616 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:34.829616 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:34.829616 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.829616 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:34.830699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:34.831419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:34.831419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:34.831419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:34.831419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:34.831419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:34.831419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:34.831419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:34.831419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.831419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:34.832073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:34.832600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:34.833103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:34.833103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.833103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:34.929913 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:34.930352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:34.930866 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:34.931274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.932151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:34.932151 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:34.932151 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:34.932151 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:34.932151 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:34.932151 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:34.932151 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.932151 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:34.933408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:34.934300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:34.934300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:34.934300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:34.934300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:34.934300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:34.934300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:34.934300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:34.934300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.934300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:34.934995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:34.935496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:34.935978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:34.935978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:35.030330 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:35.030594 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:35.030988 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:35.031235 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:35.031410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:35.031866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:44:35.031866 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:35.031866 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:35.031866 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:35.031866 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:35.031866 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:35.031866 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:35.031866 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:35.032747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:35.033632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:39.717105 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:39.718714 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:39.750472 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:39.750727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.751434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:39.751434 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:39.751434 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:39.751434 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:39.751434 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:39.751434 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:39.751434 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.751434 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.752666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.753556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:39.821859 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:39.822898 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:39.823352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:39.823663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.824710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:39.824710 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:39.824710 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:39.824710 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:39.824710 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:39.824710 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:39.824710 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.824710 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:39.825941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.826621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:39.827541 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:39.827924 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:39.828355 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:39.828559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.829156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:39.829156 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:39.829156 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:39.829156 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:39.829156 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:39.829156 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:39.829156 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.829156 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:39.829843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:39.830751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:39.830751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:39.830751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:39.830751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:39.830751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:39.830751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:39.830751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:39.830751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.830751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:39.831265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:39.832288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:39.833291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:39.833291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.833291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:39.930640 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:39.931296 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:39.932968 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:39.934005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.935317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:39.935317 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:39.935317 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:39.935317 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:39.935317 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:39.935317 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:39.935317 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.935317 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:39.936210 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:39.937004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:39.937004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:39.937004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:39.937004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:39.937004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:39.937004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:39.937004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:39.937004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.937004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:39.937421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:39.938164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:39.939022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:39.939022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:40.030858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:40.031476 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:40.032052 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:40.032318 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:40.032493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:40.033155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:44:40.033155 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:40.033155 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:40.033155 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:40.033155 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:40.033155 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:40.033155 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:40.033155 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:40.035188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:40.036075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:44.717803 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:44.720362 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:44.749077 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:44.749564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.750387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:44.750387 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:44.750387 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:44.750387 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:44.750387 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:44.750387 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:44.750387 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.750387 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.751291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.751661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:44.822541 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:44.823772 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:44.824092 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:44.824452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:44.825308 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:44.825308 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:44.825308 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:44.825308 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:44.825308 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:44.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.825308 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:44.827190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.828048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:44.828937 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:44.828937 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:44.829047 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:44.829559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.830112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:44.830112 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:44.830112 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:44.830112 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:44.830112 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:44.830112 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:44.830112 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.830112 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:44.831409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:44.831923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:44.831923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:44.831923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:44.831923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:44.831923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:44.831923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:44.831923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:44.831923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.831923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:44.832955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:44.833440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:44.833956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:44.833956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.833956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:44.931456 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:44.931832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:44.932577 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:44.932727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.933398 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:44.933398 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:44.933398 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:44.933398 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:44.933398 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:44.933398 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:44.933398 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.933398 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:44.934621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:44.935251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:44.935251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:44.935251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:44.935251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:44.935251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:44.935251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:44.935251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:44.935251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.935251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:44.935660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:44.936088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:44.936577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:44.936577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:45.031489 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:45.031851 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:45.032451 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:45.032816 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:45.033031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:45.033805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:44:45.033805 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:45.033805 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:45.033805 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:45.033805 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:45.033805 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:45.033805 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:45.033805 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:45.034941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:45.036121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:49.718511 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:49.719427 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:49.747169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:49.747638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.748461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:49.748461 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:49.748461 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:49.748461 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:49.748461 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:49.748461 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:49.748461 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.748461 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.749383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.749798 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:49.823533 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:49.824482 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:49.824802 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:49.825031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.825840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:49.825840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:49.825840 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:49.825840 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:49.825840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:49.825840 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:49.825840 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.825840 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:49.826757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.827262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:49.828424 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:49.828771 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:49.829288 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:49.829563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.830304 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:49.830304 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:49.830304 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:49.830304 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:49.830304 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:49.830304 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:49.830304 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.830304 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:49.831585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:49.831955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:49.831955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:49.831955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:49.831955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:49.831955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:49.831955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:49.831955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:49.831955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.831955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:49.833228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:49.833654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:49.834032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:49.834032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.834032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:49.932057 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:49.932574 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:49.933244 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:49.933681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.934527 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:49.934527 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:49.934527 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:49.934527 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:49.934527 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:49.934527 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:49.934527 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.934527 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:49.935364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:49.935713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:49.935713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:49.935713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:49.935713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:49.935713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:49.935713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:49.935713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:49.935713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.935713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:49.936473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:49.936962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:49.937353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:49.937353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:50.034093 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:50.034562 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:50.034994 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:50.035194 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:50.035381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:50.035920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:44:50.035920 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:50.035920 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:50.035920 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:50.035920 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:50.035920 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:50.035920 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:50.035920 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:50.036726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:50.037140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:54.719374 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:54.721121 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:54.754971 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:54.755415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.756305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:54.756305 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:54.756305 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:54.756305 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:54.756305 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:54.756305 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:54.756305 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.756305 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.757348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.758633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:54.824239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:54.825570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:54.825766 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:54.825892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.826816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:54.826816 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:54.826816 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:54.826816 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:54.826816 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:54.826816 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:54.826816 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.826816 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:54.828292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.829124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:54.830041 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:54.830041 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:54.830041 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:54.830041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.830603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:54.830603 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:54.830603 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:54.830603 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:54.830603 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:54.830603 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:54.830603 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.830603 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:54.831811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:54.832591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:54.832591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:54.832591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:54.832591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:54.832591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:54.832591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:54.832591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:54.832591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.832591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:54.833404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:54.833849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:54.834390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:54.834390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.834390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:54.932570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:54.933591 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:54.934083 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:54.934479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.935446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:54.935446 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:54.935446 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:54.935446 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:54.935446 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:54.935446 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:54.935446 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.935446 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:54.937014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:54.937676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:54.937676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:54.937676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:54.937676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:54.937676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:54.937676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:54.937676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:54.937676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.937676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:54.938728 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:54.939542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:54.940274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:54.940274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:55.034701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:55.035555 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:55.036094 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:55.036520 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:55.036756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:55.037638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:44:55.037638 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:55.037638 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:55.037638 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:55.037638 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:55.037638 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:55.037638 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:55.037638 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:55.039451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:55.040263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:59.720263 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:59.722040 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:59.756435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:59.756737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.757551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:44:59.757551 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:59.757551 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:59.757551 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:59.757551 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:59.757551 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:59.757551 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.757551 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.758751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.759677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:59.824905 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:59.825976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:59.826269 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:59.826517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.827351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:44:59.827351 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:59.827351 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:59.827351 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:59.827351 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:59.827351 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:59.827351 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.827351 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:44:59.829004 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.829904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:59.830799 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:59.830799 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:44:59.830937 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:59.831032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.831670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:44:59.831670 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:59.831670 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:59.831670 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:59.831670 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:59.831670 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:59.831670 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.831670 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:59.833236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:59.834127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:59.834127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:59.834127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:59.834127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:59.834127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:59.834127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:59.834127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:59.834127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.834127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:44:59.834989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:59.835463 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:59.835964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:59.835964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.835964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:44:59.933356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:44:59.934027 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:44:59.934734 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:44:59.935147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.936073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:44:59.936073 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:59.936073 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:44:59.936073 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:44:59.936073 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:59.936073 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:44:59.936073 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.936073 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:44:59.937486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:44:59.937899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:59.937899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:44:59.937899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:59.937899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:44:59.937899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:44:59.937899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:59.937899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:44:59.937899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.937899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:44:59.939364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:44:59.939821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:44:59.940329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:44:59.940329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:00.035400 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:00.036085 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:00.036668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:00.037078 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:00.037355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:00.038379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:45:00.038379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:00.038379 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:00.038379 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:00.038379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:00.038379 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:00.038379 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:00.038379 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:00.039682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:00.040546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:04.721026 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:04.722810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:04.760164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:04.760815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.761655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:04.761655 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:04.761655 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:04.761655 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:04.761655 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:04.761655 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:04.761655 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.761655 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.763068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.764941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:04.825589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:04.827185 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:04.827620 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:04.827863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.829171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:04.829171 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:04.829171 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:04.829171 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:04.829171 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:04.829171 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:04.829171 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.829171 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:04.830905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.831716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:04.832627 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:04.832627 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:04.832627 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:04.832627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.833375 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:04.833375 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:04.833375 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:04.833375 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:04.833375 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:04.833375 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:04.833375 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.833375 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:04.834317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:04.835347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:04.836135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.836992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:04.934036 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:04.934672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:04.935538 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:04.935952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.936912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:04.936912 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:04.936912 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:04.936912 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:04.936912 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:04.936912 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:04.936912 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.936912 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:04.937947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:04.939015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:04.939015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:04.939015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:04.939015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:04.939015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:04.939015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:04.939015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:04.939015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.939015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:04.939488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:04.940297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:04.941402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:04.941402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:05.036169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:05.037305 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:05.037891 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:05.038245 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:05.038523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:05.039353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:45:05.039353 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:05.039353 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:05.039353 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:05.039353 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:05.039353 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:05.039353 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:05.039353 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:05.041358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:05.042153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:09.721844 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:09.724760 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:09.753800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:09.754240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:09.755096 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:09.755096 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:09.755096 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:09.755096 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:09.755096 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:09.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.755096 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.756245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.757181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:09.826417 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:09.827955 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:09.828193 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:09.828369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.829589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:09.829589 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:09.829589 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:09.829589 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:09.829589 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:09.829589 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:09.829589 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.829589 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:09.832249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.833034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:09.833956 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:09.833956 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:09.834095 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:09.834355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.835192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:45:09.835192 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:09.835192 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:09.835192 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:09.835192 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:09.835192 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:09.835192 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.835192 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:09.836803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:09.837337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:09.837337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:09.837337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:09.837337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:09.837337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:09.837337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:09.837337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:09.837337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.837337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:09.838858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:09.839384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:09.839869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:09.839869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.839869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:09.934696 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:09.935558 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:09.936301 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:09.936730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.937615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:09.937615 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:09.937615 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:09.937615 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:09.937615 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:09.937615 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:09.937615 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.937615 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:09.938789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:09.939784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:09.939784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:09.939784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:09.939784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:09.939784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:09.939784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:09.939784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:09.939784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.939784 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:09.940245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:09.942070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:09.942902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:09.942902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:10.036967 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:10.037735 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:10.038346 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:10.038764 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:10.039000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:10.039860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:45:10.039860 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:10.039860 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:10.039860 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:10.039860 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:10.039860 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:10.039860 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:10.039860 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:10.041482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:10.042382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:14.722443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:14.723763 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:14.755237 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:14.755696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.756630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:14.756630 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:14.756630 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:14.756630 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:14.756630 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:14.756630 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:14.756630 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.756630 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.758005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.759497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:14.827069 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:14.828169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:14.828491 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:14.828717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.829564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:14.829564 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:14.829564 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:14.829564 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:14.829564 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:14.829564 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:14.829564 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.829564 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:14.831255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.832335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:14.833161 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:14.833857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:14.834397 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:14.834680 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.835405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:45:14.835405 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:14.835405 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:14.835405 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:14.835405 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:14.835405 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:14.835405 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.835405 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:14.836323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:14.837070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:14.837070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:14.837070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:14.837070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:14.837070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:14.837070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:14.837070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:14.837070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.837070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:14.837734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:14.838551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:14.839070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:14.839070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.839070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:14.936474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:14.936474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:14.936474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:14.936474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.937123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:14.937123 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:14.937123 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:14.937123 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:14.937123 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:14.937123 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:14.937123 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.937123 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:14.937858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:14.938667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:14.938667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:14.938667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:14.938667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:14.938667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:14.938667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:14.938667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:14.938667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.938667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:14.939086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:14.939920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:14.940983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:14.940983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:15.037572 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:15.037900 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:15.038296 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:15.038564 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:15.038662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:15.039614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:45:15.039614 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:15.039614 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:15.039614 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:15.039614 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:15.039614 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:15.039614 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:15.039614 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:15.042252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:15.042801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:19.723044 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:19.724748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:19.760116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:19.760711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.761502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:19.761502 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:19.761502 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:19.761502 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:19.761502 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:19.761502 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:19.761502 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.761502 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.762384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.763345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:19.827672 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:19.829157 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:19.829404 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:19.829548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.830322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:19.830322 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:19.830322 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:19.830322 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:19.830322 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:19.830322 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:19.830322 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.830322 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:19.831468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.832177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:19.833716 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:19.834347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:19.834826 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:19.835073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.835653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:45:19.835653 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:19.835653 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:19.835653 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:19.835653 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:19.835653 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:19.835653 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.835653 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:19.836651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:19.837622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:19.837622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:19.837622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:19.837622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:19.837622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:19.837622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:19.837622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:19.837622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.837622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:19.838084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:19.838863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:19.839649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:19.839649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.839649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:19.935814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:19.936474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:19.937762 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:19.938241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.939372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:19.939372 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:19.939372 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:19.939372 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:19.939372 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:19.939372 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:19.939372 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.939372 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:19.941038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:19.941832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:19.941832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:19.941832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:19.941832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:19.941832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:19.941832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:19.941832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:19.941832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.941832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:19.942588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:19.943465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:19.944401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:19.944401 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:20.038040 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:20.038844 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:20.039507 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:20.039884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:20.040130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:20.041101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:45:20.041101 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:20.041101 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:20.041101 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:20.041101 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:20.041101 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:20.041101 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:20.041101 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:20.043060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:20.043896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:24.723748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:24.724805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:24.758861 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:24.759548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.760482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:24.760482 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:24.760482 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:24.760482 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:24.760482 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:24.760482 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:24.760482 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.760482 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.761483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.762312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:24.828512 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:24.830048 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:24.830497 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:24.831043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.831917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:24.831917 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:24.831917 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:24.831917 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:24.831917 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:24.831917 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:24.831917 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.831917 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:24.833924 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.834739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:24.836164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:24.836164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:24.836164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:24.836164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.836563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:45:24.836563 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:24.836563 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:24.836563 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:24.836563 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:24.836563 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:24.836563 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.836563 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:24.837780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:24.838447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:24.838447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:24.838447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:24.838447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:24.838447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:24.838447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:24.838447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:24.838447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.838447 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:24.839656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:24.840250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:24.840793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:24.840793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.840793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:24.936498 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:24.943921 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:24.944427 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:24.944427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.945264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:24.945264 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:24.945264 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:24.945264 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:24.945264 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:24.945264 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:24.945264 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.945264 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:24.946207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:24.946626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:24.946626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:24.946626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:24.946626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:24.946626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:24.946626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:24.946626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:24.946626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.946626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:24.947860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:24.948307 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:24.948745 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:24.948745 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:25.038748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:25.039435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:25.040193 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:25.040701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:25.041095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:25.042026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:45:25.042026 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:25.042026 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:25.042026 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:25.042026 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:25.042026 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:25.042026 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:25.042026 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:25.044062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:25.044955 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:29.724305 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:29.725408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:29.760782 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:29.761242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.762325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:29.762325 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:29.762325 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:29.762325 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:29.762325 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:29.762325 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:29.762325 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.762325 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.763388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.764324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:29.828988 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:29.830045 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:29.830239 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:29.830385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.831493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:29.831493 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:29.831493 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:29.831493 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:29.831493 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:29.831493 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:29.831493 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.831493 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:29.832494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.833345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:29.835056 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:29.835773 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:29.835963 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:29.836037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.837301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:45:29.837301 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:29.837301 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:29.837301 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:29.837301 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:29.837301 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:29.837301 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.837301 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:29.837946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:29.838810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:29.838810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:29.838810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:29.838810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:29.838810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:29.838810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:29.838810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:29.838810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.838810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:29.839568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:29.840389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:29.841340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:29.841340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.841340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:29.943882 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:29.944481 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:29.944948 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:29.945330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.946081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:29.946081 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:29.946081 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:29.946081 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:29.946081 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:29.946081 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:29.946081 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.946081 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:29.947444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:29.948135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:29.948135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:29.948135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:29.948135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:29.948135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:29.948135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:29.948135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:29.948135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.948135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:29.949200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:29.950021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:29.950849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:29.950849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:30.039207 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:30.039852 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:30.040398 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:30.040707 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:30.040897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:30.041613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:45:30.041613 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:30.041613 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:30.041613 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:30.041613 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:30.041613 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:30.041613 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:30.041613 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:30.043441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:30.044542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:34.725145 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:34.726282 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:34.727859 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:45:34.762242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:34.762547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.763237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:34.763237 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:34.763237 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:34.763237 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:34.763237 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:34.763237 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:34.763237 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.763237 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.764439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.765264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:34.829540 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:34.830881 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:34.831127 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:34.831317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.832247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:34.832247 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:34.832247 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:34.832247 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:34.832247 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:34.832247 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:34.832247 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.832247 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:34.834452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.835877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:34.837178 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:34.837178 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:34.837178 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:34.837178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.837811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:45:34.837811 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:34.837811 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:34.837811 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:34.837811 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:34.837811 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:34.837811 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.837811 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:34.839251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:34.839775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:34.839775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:34.839775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:34.839775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:34.839775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:34.839775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:34.839775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:34.839775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.839775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:34.840836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:34.841488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:34.842261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:34.842261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.842261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:34.944447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:34.945066 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:34.945753 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:34.946147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.947052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:34.947052 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:34.947052 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:34.947052 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:34.947052 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:34.947052 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:34.947052 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.947052 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:34.948992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:34.949711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:34.949711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:34.949711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:34.949711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:34.949711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:34.949711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:34.949711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:34.949711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.949711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:34.950896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:34.951750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:34.952568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:34.952568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:35.039763 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:35.040342 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:35.040907 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:35.041396 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:35.041645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:35.042474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:45:35.042474 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:35.042474 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:35.042474 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:35.042474 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:35.042474 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:35.042474 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:35.042474 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:35.043593 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:35.044488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:39.726022 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:39.727816 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:39.760771 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:39.761189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.762056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:39.762056 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:39.762056 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:39.762056 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:39.762056 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:39.762056 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:39.762056 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.762056 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.763288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.765178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:39.830406 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:39.831546 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:39.832259 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:39.832259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.832941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:39.832941 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:39.832941 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:39.832941 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:39.832941 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:39.832941 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:39.832941 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.832941 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:39.834945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.835691 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:39.836410 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:39.836719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:39.837140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:39.837446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.838124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:45:39.838124 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:39.838124 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:39.838124 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:39.838124 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:39.838124 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:39.838124 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.838124 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:39.839485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:39.840152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:39.840152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:39.840152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:39.840152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:39.840152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:39.840152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:39.840152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:39.840152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.840152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:39.841150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:39.841831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:39.842575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:39.842575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.842575 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:39.945092 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:39.945982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:39.946871 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:39.947337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.948437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:39.948437 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:39.948437 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:39.948437 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:39.948437 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:39.948437 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:39.948437 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.948437 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:39.949960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:39.951041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:39.951041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:39.951041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:39.951041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:39.951041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:39.951041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:39.951041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:39.951041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.951041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:39.951885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:39.952950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:39.955024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:39.955024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:40.040567 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:40.045127 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:40.045747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:40.046203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:40.046502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:40.047403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:45:40.047403 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:40.047403 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:40.047403 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:40.047403 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:40.047403 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:40.047403 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:40.047403 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:40.049118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:40.050049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:44.726730 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:44.728455 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:44.763042 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:44.763296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.763640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:44.763640 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:44.763640 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:44.763640 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:44.763640 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:44.763640 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:44.763640 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.763640 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.764408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.765452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:44.831051 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:44.832424 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:44.832707 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:44.832945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.833946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:44.833946 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:44.833946 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:44.833946 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:44.833946 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:44.833946 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:44.833946 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.833946 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:44.835749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.836570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:44.837500 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:44.837500 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:44.837817 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:44.838077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.838844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:45:44.838844 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:44.838844 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:44.838844 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:44.838844 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:44.838844 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:44.838844 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.838844 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:44.840222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:44.841032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:44.841032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:44.841032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:44.841032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:44.841032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:44.841032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:44.841032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:44.841032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.841032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:44.841909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:44.842415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:44.842919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:44.842919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.842919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:44.945587 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:44.946192 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:44.946762 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:44.947130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.947957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:44.947957 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:44.947957 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:44.947957 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:44.947957 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:44.947957 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:44.947957 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.947957 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:44.949141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:44.949889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:44.949889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:44.949889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:44.949889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:44.949889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:44.949889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:44.949889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:44.949889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.949889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:44.950872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:44.951723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:44.952605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:44.952605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:45.045059 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:45.045848 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:45.046651 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:45.047149 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:45.047412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:45.048539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:45:45.048539 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:45.048539 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:45.048539 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:45.048539 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:45.048539 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:45.048539 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:45.048539 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:45.050240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:45.051023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:49.727552 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:49.729449 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:49.757890 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:49.758030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.758469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:49.758469 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:49.758469 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:49.758469 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:49.758469 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:49.758469 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:49.758469 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.758469 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.759260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.760062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:49.831680 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:49.833164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:49.833401 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:49.833747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.834759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:49.834759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:49.834759 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:49.834759 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:49.834759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:49.834759 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:49.834759 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.834759 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:49.836586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.837432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:49.838309 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:49.838309 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:49.838309 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:49.838442 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.839070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:45:49.839070 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:49.839070 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:49.839070 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:49.839070 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:49.839070 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:49.839070 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.839070 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:49.839765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:49.840299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:49.840299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:49.840299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:49.840299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:49.840299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:49.840299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:49.840299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:49.840299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.840299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:49.840591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:49.841153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:49.841686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:49.841686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.841686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:49.946118 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:49.946613 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:49.947365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:49.947772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.948751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:49.948751 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:49.948751 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:49.948751 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:49.948751 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:49.948751 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:49.948751 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.948751 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:49.949928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:49.950631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:49.950631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:49.950631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:49.950631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:49.950631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:49.950631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:49.950631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:49.950631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.950631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:49.951601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:49.952580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:49.953548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:49.953548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:50.045469 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:50.046160 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:50.046828 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:50.047238 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:50.047515 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:50.048433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Traceback (most recent call last):
Mar 23 22:45:50.048433 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:50.048433 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:50.048433 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:50.048433 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:50.048433 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:50.048433 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:50.048433 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-02e4048a-0db4-4430-8bdc-dc3507b171b0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:50.050253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:50.051073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:54.728268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:54.729884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:54.759576 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:54.759717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.760128 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:54.760128 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:54.760128 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:54.760128 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:54.760128 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:54.760128 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:54.760128 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.760128 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.760849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.761781 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:54.832368 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:54.833558 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:54.833882 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:54.834106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.835065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:54.835065 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:54.835065 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:54.835065 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:54.835065 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:54.835065 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:54.835065 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.835065 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:54.836957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.837751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:54.838721 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:54.838721 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:54.838944 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:54.839230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.840059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:45:54.840059 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:54.840059 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:54.840059 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:54.840059 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:54.840059 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:54.840059 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.840059 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:54.841001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:54.841771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:54.841771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:54.841771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:54.841771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:54.841771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:54.841771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:54.841771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:54.841771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.841771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:54.843061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:54.843911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:54.844820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:54.844820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.844820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:54.947044 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:54.947270 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:54.947841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:54.948194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.949062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:54.949062 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:54.949062 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:54.949062 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:54.949062 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:54.949062 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:54.949062 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.949062 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:54.950529 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:54.951311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:54.951311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:54.951311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:54.951311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:54.951311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:54.951311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:54.951311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:54.951311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.951311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:54.952085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:54.952572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:54.953095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:54.953095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:55.045986 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:55.046466 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:55.047252 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:55.047725 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:55.048021 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:55.049046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:45:55.049046 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:55.049046 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:55.049046 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:55.049046 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:55.049046 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:55.049046 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:55.049046 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:55.050839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:55.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:59.729012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:59.730722 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:59.767104 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:59.767688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.768665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:45:59.768665 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:59.768665 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:59.768665 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:59.768665 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:59.768665 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:59.768665 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.768665 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.769808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.770832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:59.833004 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:59.834300 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:59.834688 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:59.834974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.836054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:45:59.836054 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:59.836054 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:59.836054 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:59.836054 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:59.836054 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:59.836054 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.836054 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:45:59.838094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.838895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:59.839706 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:59.839706 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:45:59.839706 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:59.839983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.840712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:45:59.840712 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:59.840712 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:59.840712 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:59.840712 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:59.840712 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:59.840712 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.840712 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:59.841874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:59.842351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:59.842351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:59.842351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:59.842351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:59.842351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:59.842351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:59.842351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:59.842351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.842351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:45:59.843563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:59.844052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:59.844563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:59.844563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.844563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:45:59.947571 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:45:59.947935 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:45:59.948643 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:45:59.949016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.949827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:45:59.949827 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:59.949827 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:45:59.949827 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:45:59.949827 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:59.949827 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:45:59.949827 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.949827 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:45:59.951170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:45:59.951896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:59.951896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:45:59.951896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:59.951896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:45:59.951896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:45:59.951896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:59.951896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:45:59.951896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.951896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:45:59.952577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:45:59.953354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:45:59.954166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:45:59.954166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:00.046699 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:00.047349 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:00.047869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:00.048208 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:00.048474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:00.049228 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:46:00.049228 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:00.049228 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:00.049228 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:00.049228 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:00.049228 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:00.049228 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:00.049228 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:00.050797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:00.051615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:04.729718 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:04.731372 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:04.766673 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:04.767162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.767940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:04.767940 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:04.767940 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:04.767940 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:04.767940 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:04.767940 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:04.767940 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.767940 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.769326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.770366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:04.833583 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:04.834805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:04.835170 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:04.835485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.836520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:04.836520 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:04.836520 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:04.836520 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:04.836520 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:04.836520 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:04.836520 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.836520 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:04.837922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:04.839091 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:04.839356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:04.839805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:04.840084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.840910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:46:04.840910 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:04.840910 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:04.840910 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:04.840910 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:04.840910 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:04.840910 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.840910 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:04.841687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:04.842177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:04.842177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:04.842177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:04.842177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:04.842177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:04.842177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:04.842177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:04.842177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.842177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:04.842705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:04.843182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:04.843751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:04.843751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.843751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:04.948170 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:04.949111 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:04.949913 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:04.950394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.951334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:04.951334 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:04.951334 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:04.951334 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:04.951334 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:04.951334 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:04.951334 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.951334 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:04.952868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:04.953526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:04.953526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:04.953526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:04.953526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:04.953526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:04.953526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:04.953526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:04.953526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.953526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:04.954652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:04.955427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:04.956245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:04.956245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:05.047483 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:05.048263 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:05.048975 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:05.049475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:05.049720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:05.050664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:46:05.050664 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:05.050664 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:05.050664 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:05.050664 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:05.050664 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:05.050664 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:05.050664 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:05.052440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:05.053640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:09.730580 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:09.732327 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:09.764456 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:09.764824 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.765910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:09.765910 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:09.765910 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:09.765910 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:09.765910 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:09.765910 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:09.765910 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.765910 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.766654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.767359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:09.834152 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:09.835540 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:09.835802 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:09.836024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.836863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:09.836863 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:09.836863 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:09.836863 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:09.836863 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:09.836863 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:09.836863 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.836863 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:09.838559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.839310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:09.840099 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:09.840099 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:09.840268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:09.840516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.841327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:46:09.841327 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:09.841327 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:09.841327 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:09.841327 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:09.841327 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:09.841327 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.841327 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:09.842633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:09.843572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:09.843572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:09.843572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:09.843572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:09.843572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:09.843572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:09.843572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:09.843572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.843572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:09.844172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:09.844916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:09.845478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:09.845478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.845478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:09.948835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:09.949255 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:09.949954 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:09.950364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.951132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:09.951132 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:09.951132 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:09.951132 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:09.951132 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:09.951132 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:09.951132 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.951132 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:09.952421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:09.953086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:09.953086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:09.953086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:09.953086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:09.953086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:09.953086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:09.953086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:09.953086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.953086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:09.954037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:09.954836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:09.955770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:09.955770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:10.048157 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:10.048806 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:10.049498 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:10.050045 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:10.050367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:10.051178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:46:10.051178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:10.051178 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:10.051178 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:10.051178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:10.051178 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:10.051178 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:10.051178 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:10.052526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:10.053334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:14.732587 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:14.734410 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:14.768545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:14.769027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.770023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:14.770023 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:14.770023 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:14.770023 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:14.770023 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:14.770023 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:14.770023 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.770023 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.771385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.772358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:14.835276 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:14.836628 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:14.836993 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:14.837374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.838354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:14.838354 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:14.838354 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:14.838354 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:14.838354 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:14.838354 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:14.838354 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.838354 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:14.840543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.841434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:14.842589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:14.842589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:14.843113 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:14.843444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.844145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:46:14.844145 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:14.844145 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:14.844145 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:14.844145 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:14.844145 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:14.844145 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.844145 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:14.845634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:14.846108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:14.846108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:14.846108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:14.846108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:14.846108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:14.846108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:14.846108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:14.846108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.846108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:14.847241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:14.847783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:14.848327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:14.848327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.848327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:14.949534 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:14.950812 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:14.951760 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:14.952196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.953399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:14.953399 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:14.953399 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:14.953399 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:14.953399 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:14.953399 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:14.953399 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.953399 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:14.954619 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:14.955417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:14.955417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:14.955417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:14.955417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:14.955417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:14.955417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:14.955417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:14.955417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.955417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:14.956480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:14.956943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:14.957465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:14.957465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:15.049173 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:15.049782 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:15.050468 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:15.050786 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:15.050951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:15.051705 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:46:15.051705 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:15.051705 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:15.051705 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:15.051705 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:15.051705 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:15.051705 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:15.051705 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:15.053315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:15.054076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:19.733382 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:19.735061 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:19.769899 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:19.770396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.771378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:19.771378 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:19.771378 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:19.771378 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:19.771378 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:19.771378 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:19.771378 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.771378 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.772634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.773160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:19.836062 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:19.837549 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:19.837882 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:19.838149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.839259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:19.839259 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:19.839259 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:19.839259 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:19.839259 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:19.839259 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:19.839259 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.839259 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:19.840791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.841581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:19.842446 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:19.842446 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:19.842446 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:19.842661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.843519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:46:19.843519 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:19.843519 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:19.843519 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:19.843519 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:19.843519 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:19.843519 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.843519 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:19.845012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:19.845704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:19.845704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:19.845704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:19.845704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:19.845704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:19.845704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:19.845704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:19.845704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.845704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:19.846402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:19.846973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:19.847564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:19.847564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.847564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:19.950455 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:19.951066 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:19.951703 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:19.952098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.953000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:19.953000 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:19.953000 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:19.953000 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:19.953000 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:19.953000 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:19.953000 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.953000 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:19.954237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:19.954948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:19.954948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:19.954948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:19.954948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:19.954948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:19.954948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:19.954948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:19.954948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.954948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:19.955922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:19.956721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:19.957517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:19.957517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:20.049986 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:20.050619 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:20.051167 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:20.051621 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:20.051863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:20.052707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:46:20.052707 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:20.052707 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:20.052707 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:20.052707 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:20.052707 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:20.052707 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:20.052707 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:20.054372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:20.055205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:24.734192 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:24.735936 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:24.773347 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:24.773347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.774493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:24.774493 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:24.774493 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:24.774493 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:24.774493 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:24.774493 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:24.774493 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.774493 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.775376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.776293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:24.836758 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:24.838007 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:24.838386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:24.838633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.839649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:24.839649 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:24.839649 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:24.839649 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:24.839649 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:24.839649 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:24.839649 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.839649 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:24.841429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.842454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:24.843360 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:24.843360 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:24.843360 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:24.843360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.843883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:46:24.843883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:24.843883 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:24.843883 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:24.843883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:24.843883 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:24.843883 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.843883 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:24.845200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:24.846120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:24.846120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:24.846120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:24.846120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:24.846120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:24.846120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:24.846120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:24.846120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.846120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:24.846885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:24.847356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:24.847871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:24.847871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.847871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:24.951018 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:24.951903 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:24.952538 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:24.952914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.953754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:24.953754 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:24.953754 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:24.953754 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:24.953754 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:24.953754 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:24.953754 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.953754 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:24.954737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:24.955481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:24.955481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:24.955481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:24.955481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:24.955481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:24.955481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:24.955481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:24.955481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.955481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:24.955874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:24.956629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:24.957439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:24.957439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:25.050755 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:25.051109 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:25.051809 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:25.052291 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:25.052533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:25.053393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:46:25.053393 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:25.053393 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:25.053393 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:25.053393 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:25.053393 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:25.053393 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:25.053393 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:25.054967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:25.055545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:29.734920 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:29.736640 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:29.772783 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:29.773300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.774231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:29.774231 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:29.774231 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:29.774231 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:29.774231 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:29.774231 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:29.774231 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.774231 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.776177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.777589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:29.837564 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:29.838669 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:29.839103 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:29.839392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.840306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:29.840306 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:29.840306 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:29.840306 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:29.840306 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:29.840306 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:29.840306 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.840306 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:29.842075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.842928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:29.844044 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:29.844044 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:29.844044 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:29.844044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.844359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:46:29.844359 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:29.844359 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:29.844359 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:29.844359 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:29.844359 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:29.844359 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.844359 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:29.845659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:29.846074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:29.846074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:29.846074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:29.846074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:29.846074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:29.846074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:29.846074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:29.846074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.846074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:29.847406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:29.847848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:29.848423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:29.848423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.848423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:29.951684 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:29.952295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:29.953034 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:29.953371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.953981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:29.953981 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:29.953981 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:29.953981 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:29.953981 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:29.953981 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:29.953981 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.953981 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:29.954950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:29.955988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:29.955988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:29.955988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:29.955988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:29.955988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:29.955988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:29.955988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:29.955988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.955988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:29.956525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:29.958334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:29.959684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:29.959684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:30.051387 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:30.051812 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:30.052346 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:30.052771 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:30.052968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:30.053730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:46:30.053730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:30.053730 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:30.053730 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:30.053730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:30.053730 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:30.053730 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:30.053730 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:30.055295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:30.056117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:34.736066 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:34.737449 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:34.739404 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:46:34.772869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:34.773381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.774378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:34.774378 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:34.774378 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:34.774378 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:34.774378 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:34.774378 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:34.774378 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.774378 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.775537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.776591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:34.838001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:34.839187 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:34.839635 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:34.840038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.841096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:34.841096 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:34.841096 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:34.841096 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:34.841096 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:34.841096 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:34.841096 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.841096 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:34.842916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.843748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:34.845001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:34.845001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:34.845001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:34.845001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.845865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:34.845865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:34.845865 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:34.845865 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:34.845865 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:34.845865 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:34.845865 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.845865 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:34.847071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:34.847859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:34.847859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:34.847859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:34.847859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:34.847859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:34.847859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:34.847859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:34.847859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.847859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:34.848519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:34.848993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:34.849569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:34.849569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.849569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:34.952287 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:34.952731 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:34.953183 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:34.953482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.954108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:34.954108 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:34.954108 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:34.954108 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:34.954108 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:34.954108 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:34.954108 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.954108 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:34.955436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:34.956253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:34.956253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:34.956253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:34.956253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:34.956253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:34.956253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:34.956253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:34.956253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.956253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:34.957100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:34.958116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:34.958804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:34.958804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:35.051956 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:35.052402 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:35.053036 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:35.053504 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:35.053740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:35.054594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:46:35.054594 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:35.054594 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:35.054594 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:35.054594 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:35.054594 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:35.054594 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:35.054594 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:35.056241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:35.057130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:39.736622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:39.738716 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:39.764966 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:39.765124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.765632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:39.765632 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:39.765632 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:39.765632 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:39.765632 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:39.765632 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:39.765632 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.765632 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.766499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.767260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:39.839085 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:39.840611 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:39.840965 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:39.841253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.842318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:39.842318 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:39.842318 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:39.842318 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:39.842318 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:39.842318 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:39.842318 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.842318 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:39.843503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.844323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:39.845312 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:39.845312 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:39.845312 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:39.845551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.846670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:46:39.846670 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:39.846670 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:39.846670 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:39.846670 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:39.846670 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:39.846670 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.846670 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:39.848230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:39.848752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:39.848752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:39.848752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:39.848752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:39.848752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:39.848752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:39.848752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:39.848752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.848752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:39.849953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:39.851041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:39.852356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:39.852356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.852356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:39.952982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:39.953367 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:39.954106 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:39.954569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.955408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:39.955408 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:39.955408 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:39.955408 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:39.955408 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:39.955408 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:39.955408 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.955408 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:39.956317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:39.957350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:39.958261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:39.959174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:40.052677 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:40.053014 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:40.053700 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:40.053976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:40.054117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:40.054727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:46:40.054727 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:40.054727 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:40.054727 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:40.054727 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:40.054727 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:40.054727 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:40.054727 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:40.055853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:40.056374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:44.737324 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:44.738751 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:44.770308 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:44.770517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.771164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:44.771164 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:44.771164 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:44.771164 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:44.771164 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:44.771164 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:44.771164 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.771164 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.772030 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.772678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:44.839805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:44.840968 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:44.841154 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:44.841390 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.841790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:44.841790 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:44.841790 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:44.841790 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:44.841790 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:44.841790 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:44.841790 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.841790 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:44.842828 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.843369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:44.844469 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:44.844964 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:44.845281 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:44.845413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.846160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:46:44.846160 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:44.846160 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:44.846160 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:44.846160 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:44.846160 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:44.846160 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.846160 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:44.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:44.848018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:44.848018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:44.848018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:44.848018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:44.848018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:44.848018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:44.848018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:44.848018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.848018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:44.849227 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:44.849687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:44.850191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:44.850191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.850191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:44.953415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:44.953817 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:44.954464 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:44.954754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.955555 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:44.955555 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:44.955555 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:44.955555 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:44.955555 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:44.955555 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:44.955555 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.955555 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:44.957329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:44.958070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:44.958070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:44.958070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:44.958070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:44.958070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:44.958070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:44.958070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:44.958070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.958070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:44.959631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:44.960481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:44.961378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:44.961378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:45.053381 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:45.053980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:45.054606 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:45.054874 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:45.055091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:45.055706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:46:45.055706 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:45.055706 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:45.055706 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:45.055706 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:45.055706 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:45.055706 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:45.055706 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:45.057528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:45.058411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:49.738286 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:49.739977 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:49.772729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:49.772909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:49.773756 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:49.773756 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:49.773756 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:49.773756 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:49.773756 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:49.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.773756 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.775995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.776597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:49.840717 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:49.841991 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:49.842407 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:49.842665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.843804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:49.843804 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:49.843804 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:49.843804 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:49.843804 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:49.843804 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:49.843804 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.843804 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:49.845595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.846471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:49.847395 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:49.847395 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:49.847395 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:49.847395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.848066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:49.848066 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:49.848066 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:49.848066 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:49.848066 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:49.848066 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:49.848066 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.848066 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:49.849402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:49.850138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:49.850138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:49.850138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:49.850138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:49.850138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:49.850138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:49.850138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:49.850138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.850138 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:49.850897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:49.851346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:49.851890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:49.851890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.851890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:49.954196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:49.954667 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:49.955109 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:49.955377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.955939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:49.955939 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:49.955939 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:49.955939 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:49.955939 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:49.955939 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:49.955939 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.955939 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:49.956727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:49.957483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:49.957483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:49.957483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:49.957483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:49.957483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:49.957483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:49.957483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:49.957483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.957483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:49.958048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:49.959292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:49.960238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:49.960238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:50.054007 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:50.054175 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:50.054681 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:50.055014 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:50.055234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:50.055919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:46:50.055919 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:50.055919 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:50.055919 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:50.055919 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:50.055919 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:50.055919 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:50.055919 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:50.057077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:50.057570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:54.738929 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:54.740533 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:54.776951 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:54.777378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.778235 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:54.778235 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:54.778235 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:54.778235 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:54.778235 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:54.778235 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:54.778235 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.778235 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.779622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.780508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:54.841475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:54.842703 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:54.843027 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:54.843329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:54.844155 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:54.844155 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:54.844155 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:54.844155 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:54.844155 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:54.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.844155 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:54.846229 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.846933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:54.847883 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:54.847883 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:54.847883 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:54.848071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.848889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:46:54.848889 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:54.848889 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:54.848889 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:54.848889 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:54.848889 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:54.848889 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.848889 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:54.850348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:54.851093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:54.851093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:54.851093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:54.851093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:54.851093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:54.851093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:54.851093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:54.851093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.851093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:54.852058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:54.852826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:54.853664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:54.853664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.853664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:54.954653 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:54.955115 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:54.955829 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:54.956319 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.957178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:54.957178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:54.957178 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:54.957178 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:54.957178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:54.957178 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:54.957178 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.957178 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:54.958748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:54.959522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:54.959522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:54.959522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:54.959522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:54.959522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:54.959522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:54.959522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:54.959522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.959522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:54.960928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:54.961729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:54.962556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:54.962556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:55.054530 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:55.055163 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:55.055743 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:55.056147 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:55.056438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:55.057303 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:46:55.057303 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:55.057303 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:55.057303 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:55.057303 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:55.057303 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:55.057303 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:55.057303 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:55.058492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:55.058984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:59.739777 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:59.740958 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:59.772623 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:59.772790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.773200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:46:59.773200 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:59.773200 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:59.773200 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:59.773200 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:59.773200 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:59.773200 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.773200 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.773978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.775037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:59.841985 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:59.842939 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:59.843199 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:59.843500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.844205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:46:59.844205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:59.844205 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:59.844205 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:59.844205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:59.844205 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:59.844205 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.844205 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:46:59.845809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.846309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:59.846965 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:59.847722 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:46:59.848272 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:59.848556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.849250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:46:59.849250 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:59.849250 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:59.849250 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:59.849250 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:59.849250 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:59.849250 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.849250 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:59.850720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:59.851559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:59.851559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:59.851559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:59.851559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:59.851559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:59.851559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:59.851559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:59.851559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.851559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:46:59.852508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:59.853387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:59.854479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:59.854479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.854479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:46:59.955151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:46:59.955747 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:46:59.956253 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:46:59.956541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.957201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:46:59.957201 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:59.957201 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:46:59.957201 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:46:59.957201 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:59.957201 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:46:59.957201 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.957201 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:46:59.958675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:46:59.959172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:59.959172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:46:59.959172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:59.959172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:46:59.959172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:46:59.959172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:59.959172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:46:59.959172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.959172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:46:59.960257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:46:59.961040 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:46:59.962017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:46:59.962017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:00.055201 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:00.055783 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:00.056230 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:00.056675 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:00.056919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:00.057749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:00.057749 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:00.057749 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:00.057749 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:00.057749 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:00.057749 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:00.057749 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:00.057749 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:00.059582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:00.060408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:04.740371 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:04.741504 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:04.773857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:04.774145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.774857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:04.774857 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:04.774857 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:04.774857 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:04.774857 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:04.774857 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:04.774857 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.774857 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.776091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.777006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:04.842778 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:04.844033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:04.844415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:04.844685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.845662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:04.845662 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:04.845662 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:04.845662 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:04.845662 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:04.845662 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:04.845662 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.845662 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:04.847535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.848102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:04.848817 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:04.849538 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:04.849812 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:04.850022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.852720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.853382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:04.853896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:04.854410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:04.854999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:04.854999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.854999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:04.955733 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:04.956256 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:04.956850 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:04.957320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.958170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:04.958170 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:04.958170 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:04.958170 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:04.958170 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:04.958170 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:04.958170 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.958170 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:04.959095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:04.959799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:04.959799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:04.959799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:04.959799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:04.959799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:04.959799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:04.959799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:04.959799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.959799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:04.960165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:04.960886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:04.961715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:04.961715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:05.055906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:05.056392 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:05.056845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:05.057243 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:05.057468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:05.058379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:05.058379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:05.058379 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:05.058379 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:05.058379 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:05.058379 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:05.058379 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:05.058379 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:05.060031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:05.060586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:09.740961 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:09.742323 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:09.779927 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:09.780417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.781268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:09.781268 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:09.781268 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:09.781268 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:09.781268 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:09.781268 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:09.781268 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.781268 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.782157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.783060 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:09.843382 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:09.844247 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:09.844543 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:09.844767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.845531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:09.845531 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:09.845531 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:09.845531 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:09.845531 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:09.845531 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:09.845531 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.845531 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:09.847135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.847920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:09.849279 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:09.849899 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:09.850415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:09.850668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.851403 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:09.851403 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:09.851403 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:09.851403 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:09.851403 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:09.851403 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:09.851403 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.851403 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:09.852838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:09.853341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:09.853341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:09.853341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:09.853341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:09.853341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:09.853341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:09.853341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:09.853341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.853341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:09.853843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:09.854332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:09.854840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:09.854840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.854840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:09.956273 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:09.956989 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:09.957582 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:09.957930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.958736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:09.958736 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:09.958736 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:09.958736 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:09.958736 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:09.958736 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:09.958736 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.958736 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:09.960106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:09.961110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:09.961110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:09.961110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:09.961110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:09.961110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:09.961110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:09.961110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:09.961110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.961110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:09.961809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:09.962848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:09.963941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:09.963941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:10.056611 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:10.057099 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:10.057599 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:10.057957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:10.058204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:10.059097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:10.059097 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:10.059097 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:10.059097 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:10.059097 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:10.059097 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:10.059097 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:10.059097 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:10.060851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:10.061872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:14.741886 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:14.743612 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:14.777369 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:14.777638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.778414 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:14.778414 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:14.778414 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:14.778414 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:14.778414 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:14.778414 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:14.778414 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.778414 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.779637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.780589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:14.843901 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:14.844980 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:14.845308 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:14.845559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.846659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:14.846659 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:14.846659 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:14.846659 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:14.846659 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:14.846659 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:14.846659 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.846659 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:14.848665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.849497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:14.850336 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:14.850414 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:14.850920 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:14.851253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:14.852185 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:14.852185 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:14.852185 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:14.852185 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:14.852185 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:14.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.852185 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:14.853528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:14.853958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:14.853958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:14.853958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:14.853958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:14.853958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:14.853958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:14.853958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:14.853958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.853958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:14.855087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:14.855551 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:14.856020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:14.856020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.856020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:14.956857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:14.957734 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:14.958554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:14.959142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.960272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:14.960272 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:14.960272 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:14.960272 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:14.960272 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:14.960272 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:14.960272 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.960272 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:14.961422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:14.962196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:14.962196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:14.962196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:14.962196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:14.962196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:14.962196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:14.962196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:14.962196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.962196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:14.962639 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:14.963469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:14.964353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:14.964353 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:15.057140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:15.057943 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:15.058463 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:15.058805 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:15.059073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:15.059941 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:15.059941 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:15.059941 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:15.059941 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:15.059941 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:15.059941 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:15.059941 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:15.059941 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:15.061101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:15.061877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:19.742499 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:19.744105 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:19.778181 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:19.778679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.779662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:19.779662 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:19.779662 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:19.779662 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:19.779662 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:19.779662 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:19.779662 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.779662 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.780821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.781708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:19.844553 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:19.846264 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:19.846722 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:19.846983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.848089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:19.848089 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:19.848089 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:19.848089 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:19.848089 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:19.848089 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:19.848089 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.848089 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:19.850105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.850984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:19.851785 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:19.851785 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:19.852243 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:19.852570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.853431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:19.853431 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:19.853431 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:19.853431 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:19.853431 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:19.853431 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:19.853431 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.853431 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:19.854698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:19.855135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:19.855135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:19.855135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:19.855135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:19.855135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:19.855135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:19.855135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:19.855135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.855135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:19.855733 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:19.856144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:19.856629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:19.856629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.856629 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:19.957689 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:19.958364 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:19.958835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:19.959190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.960068 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:19.960068 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:19.960068 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:19.960068 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:19.960068 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:19.960068 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:19.960068 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.960068 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:19.961340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:19.962102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:19.962102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:19.962102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:19.962102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:19.962102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:19.962102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:19.962102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:19.962102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.962102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:19.962820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:19.963668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:19.964520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:19.964520 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:20.057900 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:20.058554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:20.058924 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:20.059308 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:20.059540 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:20.060241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:20.060241 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:20.060241 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:20.060241 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:20.060241 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:20.060241 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:20.060241 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:20.060241 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:20.061796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:20.062904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:24.743326 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:24.745168 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:24.779316 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:24.779522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.780251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:24.780251 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:24.780251 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:24.780251 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:24.780251 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:24.780251 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:24.780251 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.780251 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.781518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.782369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:24.845193 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:24.846547 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:24.846678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:24.846812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.847543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:24.847543 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:24.847543 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:24.847543 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:24.847543 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:24.847543 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:24.847543 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.847543 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:24.849813 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.850701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:24.851531 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:24.851812 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:24.852125 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:24.852290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.852898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:24.852898 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:24.852898 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:24.852898 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:24.852898 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:24.852898 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:24.852898 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.852898 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:24.854345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:24.854761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:24.854761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:24.854761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:24.854761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:24.854761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:24.854761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:24.854761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:24.854761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.854761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:24.856054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:24.856535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:24.857022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:24.857022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.857022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:24.958277 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:24.958849 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:24.959502 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:24.959960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.960914 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:24.960914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:24.960914 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:24.960914 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:24.960914 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:24.960914 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:24.960914 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.960914 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:24.962388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:24.963418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:24.963418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:24.963418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:24.963418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:24.963418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:24.963418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:24.963418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:24.963418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.963418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:24.964787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:24.965653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:24.966468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:24.966468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:25.058498 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:25.059048 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:25.059516 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:25.059887 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:25.060116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:25.060972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:25.060972 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:25.060972 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:25.060972 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:25.060972 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:25.060972 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:25.060972 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:25.060972 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:25.062682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:25.063480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:29.744065 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:29.745727 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:29.783198 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:29.783666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.784752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:29.784752 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:29.784752 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:29.784752 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:29.784752 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:29.784752 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:29.784752 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.784752 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.785739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.786735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:29.846021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:29.847698 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:29.848137 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:29.848429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.849701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:29.849701 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:29.849701 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:29.849701 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:29.849701 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:29.849701 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:29.849701 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.849701 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:29.851802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.852549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:29.853725 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:29.853725 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:29.853725 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:29.853725 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.854441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:29.854441 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:29.854441 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:29.854441 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:29.854441 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:29.854441 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:29.854441 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.854441 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:29.855906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:29.856617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:29.856617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:29.856617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:29.856617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:29.856617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:29.856617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:29.856617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:29.856617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.856617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:29.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:29.857716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:29.858305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:29.858305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.858305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:29.959099 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:29.959933 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:29.960840 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:29.961323 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.962329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:29.962329 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:29.962329 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:29.962329 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:29.962329 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:29.962329 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:29.962329 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.962329 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:29.963764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:29.964509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:29.964509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:29.964509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:29.964509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:29.964509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:29.964509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:29.964509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:29.964509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.964509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:29.965448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:29.966000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:29.966500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:29.966500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:30.059314 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:30.060015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:30.061207 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:30.061691 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:30.061935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:30.062957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:30.062957 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:30.062957 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:30.062957 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:30.062957 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:30.062957 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:30.062957 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:30.062957 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:30.064812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:30.065602 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:34.744811 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:34.746663 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:34.748591 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:47:34.779781 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:34.780191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.781185 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:34.781185 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:34.781185 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:34.781185 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:34.781185 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:34.781185 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:34.781185 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.781185 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.781967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.783046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:34.846768 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:34.847898 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:34.848207 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:34.848642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.849301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:34.849301 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:34.849301 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:34.849301 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:34.849301 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:34.849301 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:34.849301 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.849301 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:34.851025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.851822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:34.852642 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:34.852970 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:34.853603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:34.853872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.854666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:34.854666 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:34.854666 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:34.854666 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:34.854666 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:34.854666 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:34.854666 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.854666 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:34.855517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:34.856158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:34.856158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:34.856158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:34.856158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:34.856158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:34.856158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:34.856158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:34.856158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.856158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:34.856802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:34.857263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:34.857839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:34.857839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.857839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:34.959824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:34.960563 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:34.961163 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:34.961769 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.962479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:34.962479 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:34.962479 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:34.962479 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:34.962479 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:34.962479 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:34.962479 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.962479 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:34.964141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:34.965117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:34.965117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:34.965117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:34.965117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:34.965117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:34.965117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:34.965117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:34.965117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.965117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:34.966087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:34.966666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:34.967300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:34.967300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:35.060004 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:35.060646 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:35.061194 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:35.061601 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:35.061823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:35.062542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:35.062542 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:35.062542 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:35.062542 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:35.062542 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:35.062542 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:35.062542 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:35.062542 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:35.063926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:35.064789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:39.359513 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:39.370832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:389}}
Mar 23 22:47:39.371061 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=104232) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:370}}
Mar 23 22:47:39.371249 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Fixing resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:47:39.408713 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, resource={'id': '06cffee1-b8de-433e-a67a-ec0e950c9238', 'name': 'private', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['130adef4-306b-4397-a096-91ee5094d0e2', 'fb1912da-e258-4d41-8cd4-7896f0de6385'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:17:25Z', 'updated_at': '2026-03-23T22:17:28Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 36968}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.409185 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.409759 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c4286b-d53b-4217-a9f3-05177afa9297, external_ids_update=None, columns={'tag': 36968}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.409961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.410968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Traceback (most recent call last):
Mar 23 22:47:39.410968 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.410968 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:39.410968 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.410968 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.410968 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:39.410968 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.410968 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:39.412188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Failed to fix resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.412188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:47:39.412188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:47:39.412188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:47:39.412188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:47:39.412188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:47:39.412188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:47:39.412188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:39.412188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.412188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.412188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.412760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.413396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:47:39.413396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.413396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:47:39.413396 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Fixing resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:47:39.443867 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, resource={'id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'name': 'public', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'a53179c9-417a-4560-9bc7-e3f64551b15b'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-23T22:17:31Z', 'updated_at': '2026-03-23T22:17:40Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.444317 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.444948 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a8bd993e-d23e-4e73-8066-766307763bdb, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.445310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.446224 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Traceback (most recent call last):
Mar 23 22:47:39.446224 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.446224 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:39.446224 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.446224 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.446224 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:39.446224 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.446224 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:39.447568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Failed to fix resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.447568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:47:39.447568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:47:39.447568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:47:39.447568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:47:39.447568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:47:39.447568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:47:39.447568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:39.447568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.447568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.447568 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.448130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.448766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:47:39.448766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.448766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:47:39.448766 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Fixing resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:47:39.475971 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, resource={'id': '3a042d54-7a79-465e-bd57-47cd6833a4d0', 'name': 'admin_net', 'tenant_id': 'b4ffb2af173e4612a30978ef510f4f44', 'project_id': 'b4ffb2af173e4612a30978ef510f4f44', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['935d20c1-7d6b-489a-8358-b5fb94945b7e'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:20:57Z', 'updated_at': '2026-03-23T22:20:58Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 47875}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.476422 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.476959 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c442d0-efff-4c7e-bcb6-377b4da39c34, external_ids_update=None, columns={'tag': 47875}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.477156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.478009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Traceback (most recent call last):
Mar 23 22:47:39.478009 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.478009 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:39.478009 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.478009 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.478009 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:39.478009 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.478009 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:39.479206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Failed to fix resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.479206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:47:39.479206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:47:39.479206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:47:39.479206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:47:39.479206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:47:39.479206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:47:39.479206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:39.479206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.479206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.479206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.479547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.479940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:47:39.479940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.479940 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:47:39.479940 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Fixing resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:47:39.505730 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, resource={'id': 'c9e6c30a-b30e-473b-b06a-50be75c90b36', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['421749b7-bb0d-4854-b3f4-f0414f4c37fb'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:21:34Z', 'updated_at': '2026-03-23T22:21:35Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 48210}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.506085 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.506671 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13535482-a705-4453-af87-e8a7d8c10c82, external_ids_update=None, columns={'tag': 48210}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.506842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.507644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Traceback (most recent call last):
Mar 23 22:47:39.507644 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.507644 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:39.507644 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.507644 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.507644 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:39.507644 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.507644 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:39.508557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Failed to fix resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.508557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:47:39.508557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:47:39.508557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:47:39.508557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:47:39.508557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:47:39.508557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:47:39.508557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:39.508557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.508557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.508557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.509193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.509907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:47:39.509907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.509907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:47:39.509907 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Fixing resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:47:39.579207 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, resource={'id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'enable_snat': True}, 'gw_port_id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-t62km', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'ip_address': '172.24.5.146', 'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a'}, {'ip_address': '2001:db8::396', 'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 3, 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d'}, resource_type=routers, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.579529 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.579965 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.580211 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.580448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.581332 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Traceback (most recent call last):
Mar 23 22:47:39.581332 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.581332 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:39.581332 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.581332 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.581332 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:39.581332 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.581332 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:39.581684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Unable to update router 154c08d6-78c5-49fd-a785-0df3f4277a23. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.583010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Failed to fix resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.583010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:47:39.583010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:47:39.583010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:47:39.583010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:47:39.583010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:47:39.583010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 23 22:47:39.583010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 23 22:47:39.583010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 23 22:47:39.583010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 23 22:47:39.583010 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:47:39.583577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:47:39.584191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.584191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:47:39.584191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.584191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:47:39.584191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.584191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.584191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:47:39.584191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.584191 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:47:39.584191 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Fixing resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:47:39.693833 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, resource={'id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'name': '', 'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'project_id': '', 'mac_address': 'fa:16:3e:aa:0e:57', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160382', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.694911 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a a53179c9-417a-4560-9bc7-e3f64551b15b', 'neutron:network_name': 'neutron-bb9d5c47-4694-42c2-a018-18e3c9058915', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-154c08d6-78c5-49fd-a785-0df3f4277a23'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::396/64', '172.24.5.146/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.695557 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=348aba7c-88a8-4a19-9486-df3f7edb4007, lrouter_port=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.696016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.697067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Traceback (most recent call last):
Mar 23 22:47:39.697067 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.697067 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:39.697067 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.697067 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.697067 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:39.697067 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.697067 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:39.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Failed to fix resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:47:39.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:47:39.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:47:39.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:47:39.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:47:39.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 23 22:47:39.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:39.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.698535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.699029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.699441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:47:39.699441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.699441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:47:39.699441 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a5cc943e-b7b5-4826-b0f0-61016b26efe0 None None] Maintenance task: Synchronization completed (took 0.33 seconds)
Mar 23 22:47:39.745541 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:39.747203 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:39.781596 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.782061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.783072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:39.783072 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.783072 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:39.783072 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.783072 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.783072 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:39.783072 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.783072 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.784198 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.785124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:39.847196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:39.848414 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.848752 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.849409 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.849890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:39.849890 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.849890 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:39.849890 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.849890 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.849890 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:39.849890 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.849890 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:39.851291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.852088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:39.855537 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:39.856137 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:39.856822 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.856894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.857755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:39.857755 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.857755 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:39.857755 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.857755 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.857755 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:39.857755 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.857755 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:39.859986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:39.860503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.860503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:39.860503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.860503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:39.860503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.860503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.860503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:39.860503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.860503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:39.861963 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.862451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.862952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:39.862952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.862952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:39.960644 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:39.960994 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:39.961621 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:39.961885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.962599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:39.962599 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.962599 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:39.962599 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.962599 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.962599 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:39.962599 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.962599 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:39.964070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:39.964816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.964816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:39.964816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.964816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:39.964816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.964816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.964816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:39.964816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.964816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:39.965788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:39.966591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:39.967389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:39.967389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:40.060618 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:40.061380 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:40.062025 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:40.062502 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:40.062739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:40.063700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:40.063700 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:40.063700 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:40.063700 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:40.063700 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:40.063700 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:40.063700 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:40.063700 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:40.065591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:40.066419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:44.746768 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:44.748691 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:44.779634 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:44.780059 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.780962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:44.780962 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:44.780962 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:44.780962 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:44.780962 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:44.780962 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:44.780962 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.780962 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.781875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.782752 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:44.848074 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:44.849597 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:44.849884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:44.850539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.851347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:44.851347 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:44.851347 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:44.851347 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:44.851347 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:44.851347 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:44.851347 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.851347 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:44.853141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.853597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:44.856063 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:44.856619 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:44.857204 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:44.857519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.858087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:44.858087 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:44.858087 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:44.858087 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:44.858087 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:44.858087 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:44.858087 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.858087 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:44.859565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:44.859988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:44.859988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:44.859988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:44.859988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:44.859988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:44.859988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:44.859988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:44.859988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.859988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:44.861222 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:44.861697 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:44.862290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:44.862290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.862290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:44.961142 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:44.961439 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:44.961995 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:44.962516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.963337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:44.963337 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:44.963337 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:44.963337 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:44.963337 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:44.963337 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:44.963337 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.963337 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:44.964625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:44.964944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:44.964944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:44.964944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:44.964944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:44.964944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:44.964944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:44.964944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:44.964944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.964944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:44.966613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:44.966953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:44.967412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:44.967412 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:45.061028 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:45.061867 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:45.062555 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:45.062891 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:45.063120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:45.063986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:45.063986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:45.063986 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:45.063986 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:45.063986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:45.063986 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:45.063986 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:45.063986 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:45.065814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:45.066657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:49.747622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:49.749387 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:49.786259 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:49.786604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:49.787616 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:49.787616 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:49.787616 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:49.787616 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:49.787616 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:49.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.787616 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.789079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.789921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:49.849015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:49.849780 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:49.850076 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:49.850356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.851727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:49.851727 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:49.851727 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:49.851727 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:49.851727 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:49.851727 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:49.851727 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.851727 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:49.853933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.854508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:49.856588 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:49.856960 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:49.857534 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:49.857809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.858713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:49.858713 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:49.858713 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:49.858713 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:49.858713 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:49.858713 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:49.858713 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.858713 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:49.860045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:49.860947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:49.860947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:49.860947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:49.860947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:49.860947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:49.860947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:49.860947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:49.860947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.860947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:49.861637 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:49.862337 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:49.863066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:49.863066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.863066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:49.961637 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:49.962334 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:49.962903 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:49.963246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.963862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:49.963862 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:49.963862 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:49.963862 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:49.963862 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:49.963862 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:49.963862 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.963862 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:49.965538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:49.966153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:49.966153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:49.966153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:49.966153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:49.966153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:49.966153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:49.966153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:49.966153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.966153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:49.967380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:49.967860 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:49.968376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:49.968376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:50.061910 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:50.062974 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:50.063626 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:50.064005 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:50.064233 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:50.065273 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:50.065273 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:50.065273 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:50.065273 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:50.065273 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:50.065273 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:50.065273 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:50.065273 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:50.067152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:50.067659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:54.748459 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:54.750476 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:54.786885 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:54.787394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.788503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:54.788503 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:54.788503 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:54.788503 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:54.788503 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:54.788503 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:54.788503 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.788503 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.789807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.790888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:54.849580 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:54.851574 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:54.851994 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:54.852364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.853888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:54.853888 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:54.853888 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:54.853888 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:54.853888 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:54.853888 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:54.853888 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.853888 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:54.855613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.856076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:54.856957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:54.857623 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:54.858348 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:54.858556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.859178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:54.859178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:54.859178 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:54.859178 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:54.859178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:54.859178 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:54.859178 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.859178 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:54.860100 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:54.860945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:54.860945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:54.860945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:54.860945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:54.860945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:54.860945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:54.860945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:54.860945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.860945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:54.861439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:54.862244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:54.863301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:54.863301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.863301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:54.962699 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:54.963255 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:54.964125 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:54.964604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.965729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:54.965729 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:54.965729 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:54.965729 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:54.965729 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:54.965729 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:54.965729 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.965729 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:54.967421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:54.968439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:54.968439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:54.968439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:54.968439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:54.968439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:54.968439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:54.968439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:54.968439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.968439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:54.969012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:54.970197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:54.971548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:54.971548 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:55.062760 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:55.063467 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:55.064163 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:55.064630 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:55.064882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:55.065809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:47:55.065809 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:55.065809 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:55.065809 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:55.065809 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:55.065809 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:55.065809 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:55.065809 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:55.067711 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:55.069066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:59.749080 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:59.750962 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:59.784463 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:59.784812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.785469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:47:59.785469 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:59.785469 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:59.785469 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:59.785469 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:59.785469 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:59.785469 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.785469 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.786780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.787858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:59.850375 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:59.851468 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:59.851685 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:59.851826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.852653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:47:59.852653 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:59.852653 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:59.852653 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:59.852653 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:59.852653 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:59.852653 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.852653 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:47:59.854513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.855105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:59.857472 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:59.858089 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:47:59.858532 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:59.858700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.859331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:47:59.859331 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:59.859331 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:59.859331 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:59.859331 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:59.859331 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:59.859331 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.859331 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:59.860646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:59.861581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:59.861581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:59.861581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:59.861581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:59.861581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:59.861581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:59.861581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:59.861581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.861581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:47:59.862482 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:59.862971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:59.863517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:59.863517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.863517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:47:59.963516 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:47:59.963973 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:47:59.964869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:47:59.965474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.966449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:47:59.966449 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:59.966449 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:47:59.966449 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:47:59.966449 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:59.966449 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:47:59.966449 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.966449 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:47:59.967851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:47:59.968722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:59.968722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:47:59.968722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:59.968722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:47:59.968722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:47:59.968722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:59.968722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:47:59.968722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.968722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:47:59.969654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:47:59.970488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:47:59.971238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:47:59.971238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:00.063442 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:00.064193 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:00.064807 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:00.065424 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:00.065770 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:00.067330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:00.067330 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:00.067330 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:00.067330 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:00.067330 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:00.067330 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:00.067330 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:00.067330 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:00.068668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:00.069435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:04.749718 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:04.752723 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:04.793344 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:04.793819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.794944 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:04.794944 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:04.794944 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:04.794944 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:04.794944 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:04.794944 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:04.794944 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.794944 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.797486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.797996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:04.851253 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:04.852072 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:04.852510 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:04.852772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.853730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:04.853730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:04.853730 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:04.853730 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:04.853730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:04.853730 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:04.853730 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.853730 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:04.854974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.855428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:04.858024 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:04.858303 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:04.858829 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:04.859031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.859885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:48:04.859885 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:04.859885 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:04.859885 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:04.859885 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:04.859885 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:04.859885 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.859885 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:04.860776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:04.861592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:04.861592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:04.861592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:04.861592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:04.861592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:04.861592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:04.861592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:04.861592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.861592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:04.861892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:04.862467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:04.862991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:04.862991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.862991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:04.964042 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:04.964468 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:04.965475 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:04.965844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.967791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:48:04.967791 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:04.967791 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:04.967791 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:04.967791 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:04.967791 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:04.967791 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.967791 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:04.968755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:04.969894 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:04.970655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:04.971700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:05.064231 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:05.065019 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:05.066289 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:05.066289 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:05.066289 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:05.066883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:05.066883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:05.066883 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:05.066883 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:05.066883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:05.066883 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:05.066883 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:05.066883 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:05.068906 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:05.069710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:09.750251 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:09.751804 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:09.788950 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:09.789402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.790495 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:09.790495 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:09.790495 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:09.790495 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:09.790495 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:09.790495 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:09.790495 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.790495 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.791727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.792803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:09.851759 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:09.853335 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:09.853685 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:09.853797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.854727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:09.854727 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:09.854727 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:09.854727 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:09.854727 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:09.854727 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:09.854727 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.854727 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:09.856317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.857086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:09.858244 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:09.858947 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:09.859564 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:09.859806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.860569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:48:09.860569 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:09.860569 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:09.860569 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:09.860569 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:09.860569 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:09.860569 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.860569 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:09.862192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:09.863101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:09.863101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:09.863101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:09.863101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:09.863101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:09.863101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:09.863101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:09.863101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.863101 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:09.863597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:09.864517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:09.865875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:09.865875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.865875 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:09.964810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:09.965275 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:09.966509 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:09.967173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.968364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:48:09.968364 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:09.968364 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:09.968364 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:09.968364 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:09.968364 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:09.968364 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.968364 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:09.970174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:09.971111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:09.971111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:09.971111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:09.971111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:09.971111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:09.971111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:09.971111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.971111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:09.971538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:09.972301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:09.973089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:09.973089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:10.065060 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:10.065987 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:10.067045 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:10.067618 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:10.067876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:10.068952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:10.068952 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:10.068952 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:10.068952 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:10.068952 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:10.068952 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:10.068952 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:10.068952 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:10.070934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:10.071416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:14.751011 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:14.752719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:14.788389 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:14.789086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.789857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:14.789857 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:14.789857 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:14.789857 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:14.789857 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:14.789857 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:14.789857 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.789857 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.790766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.791775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:14.852517 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:14.853569 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:14.853792 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:14.853916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.854776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:14.854776 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:14.854776 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:14.854776 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:14.854776 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:14.854776 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:14.854776 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.854776 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:14.856701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.857467 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:14.858779 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:14.859329 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:14.859813 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:14.859936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.860511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:48:14.860511 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:14.860511 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:14.860511 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:14.860511 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:14.860511 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:14.860511 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.860511 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:14.861544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:14.862260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:14.862260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:14.862260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:14.862260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:14.862260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:14.862260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:14.862260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:14.862260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.862260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:14.863036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:14.863898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:14.864817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:14.864817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.864817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:14.965418 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:14.965759 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:14.966312 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:14.966580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.967159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:48:14.967159 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:14.967159 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:14.967159 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:14.967159 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:14.967159 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:14.967159 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.967159 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:14.968601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:14.969150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:14.969150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:14.969150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:14.969150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:14.969150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:14.969150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:14.969150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.969150 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:14.970354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:14.970869 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:14.971605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:14.971605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:15.065651 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:15.066177 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:15.066608 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:15.066837 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:15.067402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:15.067654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:15.067654 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:15.067654 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:15.067654 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:15.067654 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:15.067654 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:15.067654 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:15.067654 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:15.069487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:15.070119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:19.751773 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:19.753145 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:19.783196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:19.783659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.784501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:19.784501 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:19.784501 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:19.784501 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:19.784501 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:19.784501 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:19.784501 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.784501 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.785364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.786175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:19.853080 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:19.854365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:19.854667 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:19.854833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.855791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:19.855791 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:19.855791 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:19.855791 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:19.855791 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:19.855791 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:19.855791 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.855791 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:19.857079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.857976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:19.859060 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:19.859393 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:19.859870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:19.860163 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.860867 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:48:19.860867 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:19.860867 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:19.860867 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:19.860867 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:19.860867 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:19.860867 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.860867 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:19.861632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:19.862369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:19.862369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:19.862369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:19.862369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:19.862369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:19.862369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:19.862369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:19.862369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.862369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:19.862749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:19.863732 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:19.864836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:19.864836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.864836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:19.965869 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:19.966875 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:19.967501 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:19.967605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.968921 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:48:19.968921 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:19.968921 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:19.968921 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:19.968921 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:19.968921 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:19.968921 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.968921 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:19.969880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:19.970785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:19.970785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:19.970785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:19.970785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:19.970785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:19.970785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:19.970785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:19.970785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.970785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:19.971742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:19.972526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:19.973422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:19.973422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:20.066274 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:20.066868 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:20.067631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:20.068049 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:20.068325 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:20.069668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:20.069668 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:20.069668 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:20.069668 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:20.069668 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:20.069668 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:20.069668 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:20.069668 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:20.071379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:20.071949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:24.752473 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:24.754104 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:24.789039 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:24.789355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.789996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:24.789996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:24.789996 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:24.789996 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:24.789996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:24.789996 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:24.789996 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.789996 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.790892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.791783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:24.853858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:24.854958 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:24.855376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:24.855376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.856086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:24.856086 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:24.856086 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:24.856086 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:24.856086 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:24.856086 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:24.856086 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.856086 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:24.857042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.857572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:24.859493 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:24.859838 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:24.860431 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:24.860628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.861114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:48:24.861114 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:24.861114 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:24.861114 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:24.861114 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:24.861114 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:24.861114 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.861114 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:24.861945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:24.862420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:24.862420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:24.862420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:24.862420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:24.862420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:24.862420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:24.862420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:24.862420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.862420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:24.863143 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:24.863623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:24.864179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:24.864179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.864179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:24.966789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:24.967243 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:24.968070 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:24.968685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.969807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:48:24.969807 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:24.969807 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:24.969807 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:24.969807 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:24.969807 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:24.969807 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.969807 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:24.971708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:24.972651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:24.972651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:24.972651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:24.972651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:24.972651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:24.972651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:24.972651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:24.972651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.972651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:24.973416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:24.973958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:24.974533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:24.974533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:25.066988 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:25.067677 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:25.068352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:25.068790 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:25.068997 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:25.070029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:25.070029 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:25.070029 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:25.070029 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:25.070029 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:25.070029 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:25.070029 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:25.070029 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:25.071620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:25.072939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:29.753116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:29.755100 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:29.785554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:29.785952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.787252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:29.787252 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:29.787252 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:29.787252 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:29.787252 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:29.787252 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:29.787252 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.787252 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.788441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.789480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:29.854525 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:29.855683 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:29.856027 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:29.856312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.857149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:29.857149 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:29.857149 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:29.857149 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:29.857149 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:29.857149 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:29.857149 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.857149 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:29.858915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.859430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:29.859913 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:29.860517 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:29.860912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:29.861186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.861970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:48:29.861970 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:29.861970 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:29.861970 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:29.861970 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:29.861970 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:29.861970 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.861970 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:29.862853 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:29.863714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:29.863714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:29.863714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:29.863714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:29.863714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:29.863714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:29.863714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:29.863714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.863714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:29.863988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:29.864583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:29.865483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:29.865483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.865483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:29.967382 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:29.967824 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:29.968343 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:29.968588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.969431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Traceback (most recent call last):
Mar 23 22:48:29.969431 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:29.969431 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:29.969431 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:29.969431 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:29.969431 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:29.969431 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.969431 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:29.970786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:29.971567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:29.971567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:29.971567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:29.971567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:29.971567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:29.971567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:29.971567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:29.971567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.971567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-34c90d87-a0fb-4f58-a317-82abfab80539 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:29.972454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:29.973268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:29.974090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:29.974090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:30.067603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:30.068165 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:30.068601 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:30.068945 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:30.069167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:30.069915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:30.069915 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:30.069915 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:30.069915 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:30.069915 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:30.069915 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:30.069915 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:30.069915 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:30.071287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:30.072631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:34.753738 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:34.754807 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:34.756143 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=434 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:48:34.787372 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:34.787372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.788211 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:34.788211 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:34.788211 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:34.788211 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:34.788211 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:34.788211 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:34.788211 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.788211 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.789391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.790275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:34.855254 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:34.856561 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:34.857617 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:34.857617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.857969 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:34.857969 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:34.857969 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:34.857969 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:34.857969 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:34.857969 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:34.857969 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.857969 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:34.859569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.860140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:34.860641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:34.861019 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:34.861420 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:34.862047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.862423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:48:34.862423 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:34.862423 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:34.862423 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:34.862423 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:34.862423 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:34.862423 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.862423 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:34.863738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:34.864173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:34.864173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:34.864173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:34.864173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:34.864173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:34.864173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:34.864173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:34.864173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.864173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:34.865331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:34.865782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:34.866386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:34.866386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.866386 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:34.968095 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:34.968545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:34.969309 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:34.969610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.970280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:48:34.970280 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:34.970280 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:34.970280 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:34.970280 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:34.970280 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:34.970280 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.970280 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:34.971707 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:34.972562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:34.972562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:34.972562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:34.972562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:34.972562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:34.972562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:34.972562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.972562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:34.973358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:34.974025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:34.974626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:34.974626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:35.068388 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:35.068804 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:35.069418 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:35.069589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:35.069821 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:35.070643 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:35.070643 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:35.070643 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:35.070643 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:35.070643 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:35.070643 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:35.070643 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:35.070643 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:35.072281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:35.072820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:39.754448 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:39.756137 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:39.791263 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:39.791572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.792413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:39.792413 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:39.792413 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:39.792413 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:39.792413 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:39.792413 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:39.792413 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.792413 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.793399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.793882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:39.855830 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:39.857175 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:39.857561 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:39.857800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.858762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:39.858762 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:39.858762 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:39.858762 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:39.858762 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:39.858762 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:39.858762 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.858762 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:39.860012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.860471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:39.860972 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:39.861719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:39.862428 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:39.862627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.863308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:48:39.863308 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:39.863308 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:39.863308 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:39.863308 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:39.863308 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:39.863308 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.863308 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:39.865054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:39.866139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:39.866910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.867826 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:39.968526 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:39.969126 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:39.970062 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:39.970492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.971234 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:48:39.971234 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:39.971234 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:39.971234 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:39.971234 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:39.971234 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:39.971234 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.971234 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:39.972356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:39.972793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:39.972793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:39.972793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:39.972793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:39.972793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:39.972793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:39.972793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.972793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:39.973815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:39.974288 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:39.974902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:39.974902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:40.069021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:40.069661 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:40.070461 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:40.070919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:40.071194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:40.072088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:40.072088 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:40.072088 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:40.072088 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:40.072088 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:40.072088 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:40.072088 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:40.072088 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:40.073889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:40.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:44.755173 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:44.756862 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:44.791664 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:44.791942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.792831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:44.792831 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:44.792831 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:44.792831 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:44.792831 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:44.792831 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:44.792831 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.792831 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.794065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.795347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:44.856394 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:44.857379 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:44.857695 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:44.857928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.858823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:44.858823 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:44.858823 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:44.858823 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:44.858823 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:44.858823 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:44.858823 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.858823 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:44.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.861302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:44.862433 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:44.862433 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:44.862761 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:44.863026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.863967 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:48:44.863967 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:44.863967 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:44.863967 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:44.863967 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:44.863967 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:44.863967 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.863967 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:44.865496 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:44.866446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:44.866446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:44.866446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:44.866446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:44.866446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:44.866446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:44.866446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:44.866446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.866446 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:44.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:44.867937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:44.868652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:44.868652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.868652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:44.969133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:44.969439 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:44.970164 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:44.970617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.971395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:48:44.971395 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:44.971395 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:44.971395 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:44.971395 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:44.971395 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:44.971395 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.971395 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:44.972309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:44.973008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:44.973008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:44.973008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:44.973008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:44.973008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:44.973008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:44.973008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.973008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:44.973389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:44.974088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:44.975117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:44.975117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:45.069523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:45.070064 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:45.070715 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:45.071161 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:45.071457 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:45.072373 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:45.072373 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:45.072373 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:45.072373 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:45.072373 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:45.072373 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:45.072373 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:45.072373 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:45.074142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:45.074907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:49.756166 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:49.758079 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:49.790292 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:49.790830 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.791519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:49.791519 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:49.791519 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:49.791519 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:49.791519 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:49.791519 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:49.791519 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.791519 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.792754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.793876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:49.857207 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:49.858478 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:49.858749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:49.858983 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.859774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:49.859774 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:49.859774 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:49.859774 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:49.859774 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:49.859774 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:49.859774 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.859774 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:49.861432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.862171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:49.863002 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:49.863002 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:49.863315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:49.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.864292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:48:49.864292 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:49.864292 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:49.864292 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:49.864292 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:49.864292 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:49.864292 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.864292 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:49.865253 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:49.865975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:49.865975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:49.865975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:49.865975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:49.865975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:49.865975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:49.865975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:49.865975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.865975 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:49.866620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:49.867422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:49.868468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:49.868468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.868468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:49.969739 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:49.970146 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:49.970887 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:49.971420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.972360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:48:49.972360 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:49.972360 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:49.972360 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:49.972360 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:49.972360 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:49.972360 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.972360 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:49.973882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:49.974663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:49.974663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:49.974663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:49.974663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:49.974663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:49.974663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:49.974663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.974663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:49.975578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:49.976063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:49.976645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:49.976645 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:50.070315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:50.070935 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:50.071602 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:50.072031 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:50.072275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:50.073260 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:50.073260 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:50.073260 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:50.073260 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:50.073260 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:50.073260 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:50.073260 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:50.073260 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:50.075139 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:50.076161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:54.756814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:54.758102 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:54.794881 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:54.795308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.796147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:54.796147 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:54.796147 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:54.796147 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:54.796147 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:54.796147 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:54.796147 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.796147 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.797073 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.798244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:54.857993 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:54.859076 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:54.859642 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:54.859642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.860487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:54.860487 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:54.860487 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:54.860487 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:54.860487 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:54.860487 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:54.860487 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.860487 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:54.862135 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.863145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:54.864003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:54.864003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:54.864003 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:54.864003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:48:54.864801 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:54.864801 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:54.864801 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:54.864801 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:54.864801 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:54.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.864801 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:54.866165 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:54.867534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:54.867534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:54.867534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:54.867534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:54.867534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:54.867534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:54.867534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:54.867534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.867534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:54.867977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:54.868802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:54.869668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:54.869668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.869668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:54.970474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:54.971655 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:54.972176 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:54.972613 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.973468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:48:54.973468 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:54.973468 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:54.973468 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:54.973468 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:54.973468 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:54.973468 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.973468 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:54.974999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:54.975533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:54.975533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:54.975533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:54.975533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:54.975533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:54.975533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:54.975533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.975533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:54.976866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:54.977523 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:54.978023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:54.978023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:55.070943 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:55.071560 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:55.072232 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:55.072655 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:55.072886 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:55.073772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:48:55.073772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:55.073772 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:55.073772 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:55.073772 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:55.073772 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:55.073772 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:55.073772 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:55.075604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:55.076422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:59.757492 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:59.758884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:59.791148 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:59.791432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.792083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:48:59.792083 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:59.792083 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:59.792083 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:59.792083 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:59.792083 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:59.792083 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.792083 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.793236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.794116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:59.858506 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:59.859751 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:59.860148 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:59.860491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.861567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:59.861567 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:59.861567 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:59.861567 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:59.861567 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:59.861567 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:59.861567 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.861567 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:48:59.863553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.864357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:59.865344 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:59.865344 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:48:59.865344 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:59.865344 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.866236 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:48:59.866236 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:59.866236 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:59.866236 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:59.866236 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:59.866236 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:59.866236 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.866236 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:59.867585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:59.868385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:59.868385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:59.868385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:59.868385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:59.868385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:59.868385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:59.868385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:59.868385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.868385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:48:59.869169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:59.869635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:59.870118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:59.870118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.870118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:48:59.971250 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:48:59.971693 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:48:59.972346 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:48:59.972723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.973667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:48:59.973667 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:59.973667 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:48:59.973667 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:48:59.973667 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:59.973667 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:48:59.973667 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.973667 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:48:59.974929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:59.975516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:48:59.975516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:59.975516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:48:59.975516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:48:59.975516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:59.975516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:48:59.975516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.975516 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:48:59.976592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:48:59.977047 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:48:59.977564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:48:59.977564 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:00.071488 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:00.072152 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:00.072652 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:00.073006 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:00.073225 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:00.074091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:00.074091 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:00.074091 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:00.074091 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:00.074091 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:00.074091 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:00.074091 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:00.074091 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:00.075962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:00.076899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:04.758016 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:04.759450 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:04.806786 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:04.807050 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.807785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:04.807785 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:04.807785 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:04.807785 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:04.807785 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:04.807785 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:04.807785 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.807785 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.808768 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.809192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:04.858936 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:04.859994 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:04.860299 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:04.860530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.861547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:04.861547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:04.861547 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:04.861547 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:04.861547 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:04.861547 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:04.861547 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.861547 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:04.863584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.864410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:04.865845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:04.865845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:04.865845 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:04.865845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.866308 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:04.866308 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:04.866308 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:04.866308 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:04.866308 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:04.866308 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:04.866308 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.866308 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:04.867571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:04.868383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:04.868383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:04.868383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:04.868383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:04.868383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:04.868383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:04.868383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:04.868383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.868383 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:04.869107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:04.869692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:04.870381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:04.870381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.870381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:04.971796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:04.972415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:04.972979 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:04.973431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.974205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:04.974205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:04.974205 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:04.974205 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:04.974205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:04.974205 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:04.974205 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.974205 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:04.975538 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:04.976294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:04.976294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:04.976294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:04.976294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:04.976294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:04.976294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:04.976294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.976294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:04.977333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:04.978098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:04.978957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:04.978957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:05.072016 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:05.072570 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:05.072965 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:05.073379 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:05.073600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:05.074854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:05.074854 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:05.074854 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:05.074854 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:05.074854 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:05.074854 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:05.074854 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:05.074854 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:05.075884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:05.076460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:09.758964 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:09.759954 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:09.792954 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:09.793427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.794384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:09.794384 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:09.794384 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:09.794384 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:09.794384 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:09.794384 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:09.794384 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.794384 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.795545 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.796126 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:09.859529 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:09.860940 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:09.861244 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:09.861658 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.862740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:09.862740 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:09.862740 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:09.862740 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:09.862740 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:09.862740 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:09.862740 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.862740 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:09.864902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.865815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:09.866748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:09.866748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:09.866748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:09.866748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.867114 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:09.867114 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:09.867114 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:09.867114 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:09.867114 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:09.867114 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:09.867114 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.867114 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:09.867881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:09.868751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:09.868751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:09.868751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:09.868751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:09.868751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:09.868751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:09.868751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:09.868751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.868751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:09.869275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:09.870283 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:09.870928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:09.870928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.870928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:09.972529 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:09.973184 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:09.973689 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:09.973937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.974480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:09.974480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:09.974480 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:09.974480 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:09.974480 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:09.974480 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:09.974480 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.974480 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:09.975431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:09.976152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:09.976152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:09.976152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:09.976152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:09.976152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:09.976152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:09.976152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.976152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:09.976537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:09.977324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:09.978144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:09.978144 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:10.072594 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:10.073078 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:10.073782 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:10.074243 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:10.074597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:10.075574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:10.075574 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:10.075574 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:10.075574 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:10.075574 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:10.075574 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:10.075574 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:10.075574 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:10.077350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:10.078411 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:14.759813 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:14.761766 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:14.815196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:14.815550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.816293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:14.816293 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:14.816293 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:14.816293 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:14.816293 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:14.816293 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:14.816293 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.816293 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.817703 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.819717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:14.860256 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:14.861581 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:14.861925 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:14.862180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.863381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:14.863381 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:14.863381 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:14.863381 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:14.863381 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:14.863381 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:14.863381 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.863381 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:14.865320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.866133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:14.867067 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:14.867067 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:14.867067 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:14.867067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.867719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:14.867719 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:14.867719 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:14.867719 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:14.867719 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:14.867719 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:14.867719 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.867719 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:14.869249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:14.870032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:14.870032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:14.870032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:14.870032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:14.870032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:14.870032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:14.870032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:14.870032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.870032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:14.871056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:14.871836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:14.872471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:14.872471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.872471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:14.973494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:14.974367 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:14.975200 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:14.975693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.976730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:14.976730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:14.976730 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:14.976730 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:14.976730 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:14.976730 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:14.976730 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.976730 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:14.978327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:14.979082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:14.979082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:14.979082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:14.979082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:14.979082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:14.979082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:14.979082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.979082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:14.980056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:14.980915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:14.981817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:14.981817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:15.073399 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:15.074248 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:15.074801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:15.075040 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:15.075251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:15.075909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:15.075909 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:15.075909 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:15.075909 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:15.075909 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:15.075909 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:15.075909 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:15.075909 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:15.077841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:15.078881 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:19.760656 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:19.762465 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:19.795095 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:19.795522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.796384 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:19.796384 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:19.796384 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:19.796384 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:19.796384 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:19.796384 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:19.796384 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.796384 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.797454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.798329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:19.861026 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:19.862072 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:19.862259 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:19.862399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.863322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:19.863322 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:19.863322 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:19.863322 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:19.863322 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:19.863322 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:19.863322 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.863322 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:19.864609 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.865113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:19.867334 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:19.867334 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:19.867750 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:19.867999 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.868954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:19.868954 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:19.868954 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:19.868954 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:19.868954 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:19.868954 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:19.868954 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.868954 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:19.870578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:19.870916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:19.870916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:19.870916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:19.870916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:19.870916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:19.870916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:19.870916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:19.870916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.870916 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:19.872287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:19.872605 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:19.872948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:19.872948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.872948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:19.974365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:19.974857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:19.975478 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:19.975948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.977115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:19.977115 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:19.977115 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:19.977115 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:19.977115 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:19.977115 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:19.977115 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.977115 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:19.978578 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:19.979574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:19.979574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:19.979574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:19.979574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:19.979574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:19.979574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:19.979574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.979574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:19.980104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:19.981120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:19.982243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:19.982243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:20.074076 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:20.074982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:20.075571 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:20.075989 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:20.076269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:20.077558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:20.077558 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:20.077558 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:20.077558 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:20.077558 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:20.077558 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:20.077558 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:20.077558 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:20.079015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:20.079700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:24.761474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:24.762759 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:24.798286 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:24.798557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.799274 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:24.799274 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:24.799274 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:24.799274 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:24.799274 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:24.799274 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:24.799274 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.799274 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.800772 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.801864 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:24.861489 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:24.862729 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:24.863103 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:24.863404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.864582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:24.864582 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:24.864582 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:24.864582 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:24.864582 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:24.864582 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:24.864582 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.864582 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:24.865731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.866673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:24.867528 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:24.867528 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:24.868093 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:24.868524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.868996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:24.868996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:24.868996 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:24.868996 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:24.868996 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:24.868996 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:24.868996 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.868996 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:24.869973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:24.870666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:24.870666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:24.870666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:24.870666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:24.870666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:24.870666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:24.870666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:24.870666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.870666 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:24.871026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:24.871682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:24.872454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:24.872454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.872454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:24.975008 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:24.975774 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:24.976519 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:24.976934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.977755 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:24.977755 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:24.977755 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:24.977755 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:24.977755 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:24.977755 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:24.977755 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.977755 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:24.978694 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:24.979422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:24.979422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:24.979422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:24.979422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:24.979422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:24.979422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:24.979422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.979422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:24.979759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:24.980422 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:24.980908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:24.980908 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:25.074701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:25.075486 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:25.076433 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:25.076868 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:25.077116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:25.077907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:25.077907 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:25.077907 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:25.077907 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:25.077907 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:25.077907 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:25.077907 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:25.077907 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:25.079773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:25.080698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:29.762071 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:29.763290 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:29.795764 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:29.796162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.797070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:29.797070 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:29.797070 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:29.797070 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:29.797070 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:29.797070 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:29.797070 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.797070 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.798188 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.799023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:29.862148 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:29.863698 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:29.864067 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:29.864356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.865679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:29.865679 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:29.865679 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:29.865679 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:29.865679 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:29.865679 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:29.865679 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.865679 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:29.868347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.869097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:29.869896 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:29.869896 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:29.869896 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:29.869896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.870641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:29.870641 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:29.870641 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:29.870641 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:29.870641 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:29.870641 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:29.870641 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.870641 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:29.872202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:29.873080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:29.873080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:29.873080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:29.873080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:29.873080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:29.873080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:29.873080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:29.873080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.873080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:29.873734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:29.874488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:29.875342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:29.875342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.875342 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:29.975814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:29.976451 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:29.977303 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:29.977744 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.978649 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:29.978649 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:29.978649 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:29.978649 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:29.978649 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:29.978649 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:29.978649 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.978649 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:29.980372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:29.981041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:29.981041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:29.981041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:29.981041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:29.981041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:29.981041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:29.981041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.981041 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:29.982065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:29.982835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:29.983686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:29.983686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:30.075367 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:30.076011 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:30.076598 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:30.076966 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:30.077400 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:30.077993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:30.077993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:30.077993 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:30.077993 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:30.077993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:30.077993 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:30.077993 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:30.077993 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:30.079726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:30.080585 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:34.762795 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:34.764379 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:34.766047 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:49:34.801298 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:34.801574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.802270 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:34.802270 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:34.802270 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:34.802270 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:34.802270 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:34.802270 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:34.802270 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.802270 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.803363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.804118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:34.862859 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:34.864383 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:34.864828 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:34.865058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.866055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:34.866055 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:34.866055 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:34.866055 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:34.866055 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:34.866055 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:34.866055 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.866055 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:34.867902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.868792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:34.869622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:34.869622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:34.869984 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:34.870326 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.871028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:34.871028 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:34.871028 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:34.871028 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:34.871028 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:34.871028 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:34.871028 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.871028 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:34.872460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:34.873268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:34.873268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:34.873268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:34.873268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:34.873268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:34.873268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:34.873268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:34.873268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.873268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:34.874056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:34.874802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:34.875300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:34.875300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.875300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:34.976583 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:34.977033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:34.977923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:34.978377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.979148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:34.979148 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:34.979148 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:34.979148 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:34.979148 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:34.979148 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:34.979148 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.979148 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:34.980032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:34.980553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:34.980553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:34.980553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:34.980553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:34.980553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:34.980553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:34.980553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.980553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:34.981321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:34.981829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:34.982559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:34.982559 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:35.076090 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:35.076842 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:35.077843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:35.078032 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:35.078292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:35.079345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:35.079345 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:35.079345 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:35.079345 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:35.079345 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:35.079345 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:35.079345 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:35.079345 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:35.080832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:35.081572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:39.763848 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:39.766063 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:39.840151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:39.840580 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.841528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:39.841528 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:39.841528 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:39.841528 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:39.841528 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:39.841528 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:39.841528 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.841528 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.842474 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.843987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:39.863503 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:39.864595 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:39.864821 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:39.865032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.865809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:39.865809 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:39.865809 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:39.865809 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:39.865809 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:39.865809 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:39.865809 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.865809 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:39.867584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.868786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:39.870032 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:39.870793 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:39.871614 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:39.872035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.872721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:39.872721 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:39.872721 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:39.872721 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:39.872721 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:39.872721 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:39.872721 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.872721 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:39.873739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:39.874662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:39.874662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:39.874662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:39.874662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:39.874662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:39.874662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:39.874662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:39.874662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.874662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:39.875293 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:39.875935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:39.876592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:39.876592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.876592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:39.977191 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:39.977919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:39.979276 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:39.979554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.980699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:39.980699 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:39.980699 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:39.980699 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:39.980699 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:39.980699 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:39.980699 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.980699 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:39.982026 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:39.983151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:39.983151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:39.983151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:39.983151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:39.983151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:39.983151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:39.983151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.983151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:39.983754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:39.984893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:39.986410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:39.986410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:40.076616 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:40.077176 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:40.077783 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:40.078031 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:40.078153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:40.078742 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:40.078742 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:40.078742 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:40.078742 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:40.078742 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:40.078742 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:40.078742 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:40.078742 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:40.080301 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:40.080812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:44.764583 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:44.766350 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:44.803096 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:44.803535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.804429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:44.804429 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:44.804429 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:44.804429 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:44.804429 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:44.804429 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:44.804429 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.804429 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.805623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.806120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:44.864179 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:44.865567 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:44.865905 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:44.866148 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.867389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:44.867389 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:44.867389 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:44.867389 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:44.867389 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:44.867389 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:44.867389 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.867389 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:44.869553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.870195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:44.871063 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:44.871063 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:44.871063 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:44.871452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.872361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:44.872361 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:44.872361 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:44.872361 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:44.872361 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:44.872361 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:44.872361 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.872361 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:44.873681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:44.874103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:44.874103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:44.874103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:44.874103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:44.874103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:44.874103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:44.874103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:44.874103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.874103 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:44.875488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:44.875961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:44.876503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:44.876503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.876503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:44.978114 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:44.978763 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:44.979546 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:44.979974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.980812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:44.980812 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:44.980812 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:44.980812 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:44.980812 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:44.980812 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:44.980812 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.980812 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:44.981806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:44.982563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:44.982563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:44.982563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:44.982563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:44.982563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:44.982563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:44.982563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.982563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:44.983046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:44.983779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:44.984577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:44.984577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:45.077197 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:45.078081 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:45.078553 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:45.078840 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:45.078993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:45.079931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:45.079931 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:45.079931 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:45.079931 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:45.079931 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:45.079931 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:45.079931 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:45.079931 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:45.081888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:45.082806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:49.765464 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:49.767726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:49.802307 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:49.802712 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.804015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:49.804015 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:49.804015 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:49.804015 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:49.804015 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:49.804015 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:49.804015 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.804015 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.805245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.806263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:49.864821 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:49.866797 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:49.867151 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:49.867535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.869316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:49.869316 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:49.869316 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:49.869316 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:49.869316 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:49.869316 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:49.869316 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.869316 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:49.871561 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.872837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:49.874365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:49.874365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:49.874365 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:49.874365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.874787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:49.874787 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:49.874787 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:49.874787 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:49.874787 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:49.874787 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:49.874787 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.874787 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:49.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:49.876404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:49.876404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:49.876404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:49.876404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:49.876404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:49.876404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:49.876404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:49.876404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.876404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:49.877539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:49.877987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:49.878483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:49.878483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.878483 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:49.978863 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:49.979766 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:49.980441 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:49.980891 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.981743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:49.981743 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:49.981743 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:49.981743 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:49.981743 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:49.981743 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:49.981743 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.981743 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:49.982868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:49.983647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:49.983647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:49.983647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:49.983647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:49.983647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:49.983647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:49.983647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.983647 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:49.984009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:49.984796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:49.985630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:49.985630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:50.077870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:50.078765 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:50.079474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:50.079847 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:50.080086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:50.081051 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:50.081051 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:50.081051 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:50.081051 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:50.081051 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:50.081051 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:50.081051 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:50.081051 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:50.083280 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:50.084065 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:54.766286 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:54.768420 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:54.801632 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:54.801998 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.803025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:54.803025 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:54.803025 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:54.803025 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:54.803025 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:54.803025 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:54.803025 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.803025 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.804055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.805354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:54.865497 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:54.866842 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:54.867259 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:54.867589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.868597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:54.868597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:54.868597 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:54.868597 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:54.868597 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:54.868597 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:54.868597 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.868597 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:54.870246 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.870978 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:54.872477 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:54.873187 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:54.873885 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:54.874175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.875152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:54.875152 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:54.875152 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:54.875152 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:54.875152 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:54.875152 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:54.875152 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.875152 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:54.876722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:54.877572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:54.877572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:54.877572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:54.877572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:54.877572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:54.877572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:54.877572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:54.877572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.877572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:54.878416 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:54.879182 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:54.880132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:54.880132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.880132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:54.979719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:54.980414 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:54.981119 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:54.981653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.982554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:54.982554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:54.982554 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:54.982554 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:54.982554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:54.982554 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:54.982554 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.982554 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:54.983793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:54.984557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:54.984557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:54.984557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:54.984557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:54.984557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:54.984557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:54.984557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.984557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:54.985583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:54.986381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:54.987152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:54.987152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:55.078554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:55.079192 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:55.079760 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:55.080254 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:55.080625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:55.081710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:49:55.081710 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:55.081710 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:55.081710 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:55.081710 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:55.081710 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:55.081710 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:55.081710 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:55.082903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:55.083449 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:59.767207 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:59.768880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:59.809133 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:59.809672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.810688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:49:59.810688 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:59.810688 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:59.810688 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:59.810688 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:59.810688 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:59.810688 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.810688 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.811600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.812584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:59.866021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:59.867167 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:59.867610 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:59.867934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.868910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:49:59.868910 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:59.868910 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:59.868910 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:59.868910 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:59.868910 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:59.868910 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.868910 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:49:59.870918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.871787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:59.873136 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:59.873749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:49:59.874321 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:59.875061 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.875720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:49:59.875720 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:59.875720 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:59.875720 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:59.875720 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:59.875720 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:59.875720 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.875720 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:59.876935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:59.877825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:59.877825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:59.877825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:59.877825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:59.877825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:59.877825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:59.877825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:59.877825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.877825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:49:59.878497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:59.879354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:59.880249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:59.880249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.880249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:49:59.980251 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:49:59.980554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:49:59.981082 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:49:59.981484 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.982369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:49:59.982369 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:59.982369 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:49:59.982369 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:49:59.982369 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:59.982369 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:49:59.982369 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.982369 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:49:59.983364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:59.984094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:49:59.984094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:59.984094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:49:59.984094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:49:59.984094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:59.984094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:49:59.984094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.984094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:49:59.984513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:49:59.985406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:49:59.986321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:49:59.986321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:00.079140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:00.079708 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:00.080285 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:00.080664 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:00.080890 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:00.081803 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:00.081803 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:00.081803 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:00.081803 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:00.081803 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:00.081803 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:00.081803 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:00.081803 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:00.083534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:00.084365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:04.767858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:04.769085 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:04.804137 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:04.804408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.805181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:04.805181 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:04.805181 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:04.805181 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:04.805181 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:04.805181 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:04.805181 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.805181 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.806408 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.807465 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:04.866731 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:04.868030 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:04.868432 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:04.868696 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.869815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:04.869815 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:04.869815 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:04.869815 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:04.869815 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:04.869815 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:04.869815 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.869815 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:04.871630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.872849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:04.874078 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:04.874276 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:04.874876 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:04.875172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.875930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:04.875930 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:04.875930 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:04.875930 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:04.875930 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:04.875930 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:04.875930 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.875930 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:04.876872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:04.877577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:04.877577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:04.877577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:04.877577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:04.877577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:04.877577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:04.877577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:04.877577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.877577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:04.878455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:04.879055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:04.879787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:04.879787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.879787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:04.994428 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:04.994428 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:04.994645 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:04.994909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.995500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:50:04.995500 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:04.995500 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:04.995500 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:04.995500 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:04.995500 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:04.995500 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.995500 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:04.996232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:04.996539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:04.996539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:04.996539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:04.996539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:04.996539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:04.996539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:04.996539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.996539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:50:04.997119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:04.997806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:04.998173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:04.998173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:05.079865 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:05.080726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:05.081508 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:05.081957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:05.082292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:05.083162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:05.083162 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:05.083162 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:05.083162 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:05.083162 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:05.083162 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:05.083162 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:05.083162 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:05.084918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:05.085759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:09.768603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:09.770392 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:09.800651 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:09.800804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.801195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:09.801195 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:09.801195 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:09.801195 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:09.801195 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:09.801195 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:09.801195 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.801195 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:09.801954 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.802607 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:09.867423 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:09.868818 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:09.869083 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:09.869369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.870343 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:09.870343 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:09.870343 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:09.870343 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:09.870343 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:09.870343 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:09.870343 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.870343 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:09.872087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.872964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:09.874141 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:09.874454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:09.874960 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:09.875181 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:09.875893 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:09.875893 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:09.875893 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:09.875893 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:09.875893 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:09.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.875893 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:09.876727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:09.877248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:09.877248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:09.877248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:09.877248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:09.877248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:09.877248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:09.877248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:09.877248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.877248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:09.877716 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:09.878268 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:09.878778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:09.878778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.878778 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:09.994493 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:09.994850 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:09.995131 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:09.995379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.995792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:50:09.995792 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:09.995792 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:09.995792 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:09.995792 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:09.995792 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:09.995792 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:09.995792 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:10.048747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:10.049665 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:10.050071 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:10.050731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:10.050731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:10.050731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:10.050731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:10.050731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:10.050731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:10.050731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:10.050731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:10.050731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:10.080576 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:10.081479 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:10.082168 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:10.082582 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:10.082811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:10.083928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:10.083928 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:10.083928 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:10.083928 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:10.083928 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:10.083928 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:10.083928 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:10.083928 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:10.085910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:10.086434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:14.769246 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:14.770987 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:14.804379 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:14.804503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.805201 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:14.805201 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:14.805201 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:14.805201 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:14.805201 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:14.805201 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:14.805201 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.805201 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:14.806505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.807427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:14.867959 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:14.869141 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:14.869497 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:14.869731 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.870773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:14.870773 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:14.870773 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:14.870773 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:14.870773 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:14.870773 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:14.870773 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.870773 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:14.872581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.873478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:14.874601 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:14.875247 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:14.875889 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:14.876151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.876945 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:14.876945 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:14.876945 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:14.876945 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:14.876945 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:14.876945 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:14.876945 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.876945 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:14.878537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:14.879345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:14.879345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:14.879345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:14.879345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:14.879345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:14.879345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:14.879345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:14.879345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.879345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:14.880817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:14.881299 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:14.881797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:14.881797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.881797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:14.995196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:14.996434 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:14.996984 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:14.997372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.998166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:50:14.998166 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:14.998166 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:14.998166 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:14.998166 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:14.998166 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:14.998166 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.998166 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:14.999863 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:15.000634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:15.000634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:15.000634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:15.000634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:15.000634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:15.000634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:15.000634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:15.000634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:50:15.001008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:15.001748 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:15.002587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:15.002587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:15.081363 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:15.082060 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:15.082767 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:15.083180 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:15.083448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:15.084404 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:15.084404 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:15.084404 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:15.084404 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:15.084404 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:15.084404 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:15.084404 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:15.084404 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:15.086118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:15.087302 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:19.770035 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:19.771408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:19.805762 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:19.806158 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.806812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:19.806812 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:19.806812 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:19.806812 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:19.806812 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:19.806812 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:19.806812 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.806812 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:19.807698 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.808291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:19.868595 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:19.869669 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:19.869996 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:19.870238 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.871296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:19.871296 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:19.871296 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:19.871296 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:19.871296 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:19.871296 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:19.871296 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.871296 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:19.872974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.873858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:19.875236 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:19.875866 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:19.876360 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:19.876626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.877948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:19.877948 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:19.877948 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:19.877948 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:19.877948 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:19.877948 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:19.877948 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.877948 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:19.878870 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:19.879395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:19.879395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:19.879395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:19.879395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:19.879395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:19.879395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:19.879395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:19.879395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.879395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:19.879857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:19.880415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:19.880893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:19.880893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.880893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:19.996492 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:19.997578 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:19.998695 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:19.999056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.999986 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:50:19.999986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:19.999986 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:19.999986 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:19.999986 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:19.999986 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:19.999986 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:19.999986 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:20.001079 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:20.001929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:20.001929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:20.001929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:20.001929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:20.001929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:20.001929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:20.001929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:20.001929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:50:20.002763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:20.003721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:20.004615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:20.004615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:20.081843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:20.082342 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:20.083035 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:20.083603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:20.083722 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:20.084738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:20.084738 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:20.084738 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:20.084738 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:20.084738 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:20.084738 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:20.084738 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:20.084738 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:20.086240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:20.086987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:24.770705 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:24.772440 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:24.808357 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:24.808811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.809710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:24.809710 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:24.809710 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:24.809710 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:24.809710 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:24.809710 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:24.809710 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.809710 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:24.810633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.811371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:24.869187 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:24.870408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:24.870718 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:24.870953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.871879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:24.871879 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:24.871879 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:24.871879 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:24.871879 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:24.871879 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:24.871879 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.871879 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:24.874207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.874965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:24.875951 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:24.876416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:24.876879 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:24.877136 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.877985 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:24.877985 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:24.877985 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:24.877985 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:24.877985 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:24.877985 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:24.877985 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.877985 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:24.879699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:24.880917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:24.880917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:24.880917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:24.880917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:24.880917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:24.880917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:24.880917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:24.880917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.880917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:24.881754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:24.882297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:24.882845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:24.882845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:24.882845 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:25.012315 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:25.013172 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:25.014031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:25.014480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:25.014965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:25.014965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:25.014965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:25.014965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:25.014965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:25.014965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:25.014965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:25.014965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:25.014965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:25.082445 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:25.083062 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:25.083536 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:25.083879 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:25.084096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:25.084893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:25.084893 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:25.084893 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:25.084893 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:25.084893 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:25.084893 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:25.084893 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:25.084893 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:25.086552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:25.087356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:29.771482 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:29.772520 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:29.807116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:29.807333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.807973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:29.807973 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:29.807973 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:29.807973 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:29.807973 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:29.807973 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:29.807973 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.807973 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:29.809110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.809929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:29.869716 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:29.871044 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:29.871428 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:29.871677 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.872673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:29.872673 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:29.872673 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:29.872673 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:29.872673 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:29.872673 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:29.872673 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.872673 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:29.874521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.875355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:29.876764 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:29.877234 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:29.877778 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:29.878024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.878903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:29.878903 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:29.878903 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:29.878903 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:29.878903 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:29.878903 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:29.878903 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.878903 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:29.880190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:29.880610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:29.880610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:29.880610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:29.880610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:29.880610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:29.880610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:29.880610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:29.880610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.880610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:29.881787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:29.882266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:29.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:29.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:29.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:29.998943 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:29.999401 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:30.000356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:30.000571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:30.001311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:50:30.001311 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:30.001311 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:30.001311 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:30.001311 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:30.001311 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:30.001311 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:30.001311 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:30.002066 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:30.002918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:30.002918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:30.002918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:30.002918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:30.002918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:30.002918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:30.002918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:30.002918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:50:30.003287 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:30.004330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:30.005109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:30.005109 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:30.083260 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:30.083883 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:30.084447 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:30.084769 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:30.084901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:30.085583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:30.085583 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:30.085583 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:30.085583 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:30.085583 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:30.085583 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:30.085583 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:30.085583 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:30.087472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:30.088042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:34.772208 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:34.773790 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:34.775750 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:50:34.816125 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:34.816667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.818196 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:34.818196 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:34.818196 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:34.818196 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:34.818196 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:34.818196 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:34.818196 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.818196 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:34.819355 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.820089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:34.870415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:34.871511 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:34.871730 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:34.871887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.872915 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:34.872915 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:34.872915 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:34.872915 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:34.872915 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:34.872915 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:34.872915 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.872915 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:34.873895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.874382 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:34.877232 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:34.877790 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:34.878435 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:34.878435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.878928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:34.878928 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:34.878928 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:34.878928 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:34.878928 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:34.878928 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:34.878928 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.878928 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:34.880242 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:34.880810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:34.880810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:34.880810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:34.880810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:34.880810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:34.880810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:34.880810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:34.880810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.880810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:34.882018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:34.882552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:34.883085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:34.883085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:34.883085 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:35.000800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:35.000800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:35.001397 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:35.001793 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:35.002922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:50:35.002922 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:35.002922 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:35.002922 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:35.002922 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:35.002922 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:35.002922 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:35.002922 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:35.003754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:35.004294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:35.004294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:35.004294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:35.004294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:35.004294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:35.004294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:35.004294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:35.004294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:50:35.005389 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:35.005899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:35.006421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:35.006421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:35.084373 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:35.085153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:35.085664 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:35.085955 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:35.086093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:35.086721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:35.086721 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:35.086721 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:35.086721 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:35.086721 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:35.086721 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:35.086721 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:35.086721 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:35.087799 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:35.088909 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:39.772875 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:39.774267 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:39.846529 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:39.846992 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.847993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:39.847993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:39.847993 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:39.847993 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:39.847993 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:39.847993 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:39.847993 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.847993 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:39.849819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.850601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:39.871142 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:39.872190 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:39.872506 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:39.872740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.873546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:39.873546 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:39.873546 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:39.873546 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:39.873546 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:39.873546 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:39.873546 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.873546 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:39.875183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.875950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:39.877735 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:39.878068 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:39.878530 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:39.878779 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.879502 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:39.879502 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:39.879502 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:39.879502 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:39.879502 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:39.879502 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:39.879502 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.879502 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:39.880318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:39.880753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:39.880753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:39.880753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:39.880753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:39.880753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:39.880753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:39.880753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:39.880753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.880753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:39.881331 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:39.881773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:39.882281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:39.882281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:39.882281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:40.000706 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:40.001184 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:40.001922 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:40.002448 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:40.003195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:50:40.003195 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:40.003195 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:40.003195 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:40.003195 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:40.003195 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:40.003195 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:40.003195 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:40.004123 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:40.004897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:40.004897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:40.004897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:40.004897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:40.004897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:40.004897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:40.004897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:40.004897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:50:40.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:40.006003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:40.006970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:40.006970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:40.085007 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:40.086112 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:40.087037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:40.087376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:40.087547 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:40.088393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:40.088393 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:40.088393 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:40.088393 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:40.088393 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:40.088393 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:40.088393 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:40.088393 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:40.090479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:40.090877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:44.773689 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:44.775640 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:44.812380 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:44.812775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.813672 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:44.813672 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:44.813672 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:44.813672 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:44.813672 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:44.813672 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:44.813672 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.813672 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:44.814595 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.815701 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:44.872018 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:44.873565 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:44.873835 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:44.874054 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.874806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:44.874806 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:44.874806 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:44.874806 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:44.874806 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:44.874806 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:44.874806 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.874806 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:44.876402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.876882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:44.878405 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:44.878933 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:44.879514 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:44.880031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.881137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:44.881137 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:44.881137 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:44.881137 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:44.881137 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:44.881137 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:44.881137 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.881137 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:44.882435 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:44.882902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:44.882902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:44.882902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:44.882902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:44.882902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:44.882902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:44.882902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:44.882902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.882902 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:44.883919 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:44.884379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:44.884848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:44.884848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:44.884848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:45.001408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:45.002253 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:45.003125 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:45.003676 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:45.004653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:50:45.004653 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:45.004653 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:45.004653 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:45.004653 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:45.004653 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:45.004653 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:45.004653 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:45.006194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:45.007072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:45.007072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:45.007072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:45.007072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:45.007072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:45.007072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:45.007072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:45.007072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:50:45.008096 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:45.009016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:45.009973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:45.009973 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:45.085832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:45.086563 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:45.087174 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:45.087601 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:45.087854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:45.088880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:45.088880 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:45.088880 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:45.088880 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:45.088880 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:45.088880 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:45.088880 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:45.088880 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:45.090636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:45.091371 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:49.774332 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:49.775295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:49.809973 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:49.810735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.811095 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:49.811095 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:49.811095 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:49.811095 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:49.811095 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:49.811095 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:49.811095 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.811095 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:49.812324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.813930 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:49.873084 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:49.874028 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:49.874320 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:49.874544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.875321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:49.875321 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:49.875321 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:49.875321 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:49.875321 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:49.875321 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:49.875321 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.875321 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:49.876939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.877877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:49.878846 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:49.879782 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:49.880320 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:49.880574 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.881247 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:49.881247 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:49.881247 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:49.881247 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:49.881247 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:49.881247 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:49.881247 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.881247 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:49.882193 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:49.882960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:49.882960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:49.882960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:49.882960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:49.882960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:49.882960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:49.882960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:49.882960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.882960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:49.883476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:49.884327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:49.885445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:49.885445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:49.885445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:50.002149 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:50.002801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:50.003437 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:50.003834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:50.004630 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:50:50.004630 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:50.004630 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:50.004630 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:50.004630 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:50.004630 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:50.004630 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:50.004630 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:50.005965 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:50.006761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:50.006761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:50.006761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:50.006761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:50.006761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:50.006761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:50.006761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:50.006761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:50:50.007693 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:50.008504 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:50.009399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:50.009399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:50.086616 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:50.087039 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:50.087638 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:50.088116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:50.088266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:50.089119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:50.089119 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:50.089119 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:50.089119 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:50.089119 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:50.089119 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:50.089119 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:50.089119 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:50.090739 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:50.091771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:54.774996 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:54.776722 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:54.809670 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:54.809950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.810843 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:54.810843 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:54.810843 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:54.810843 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:54.810843 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:54.810843 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:54.810843 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.810843 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:54.811750 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.812706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:54.873691 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:54.874767 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:54.874889 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:54.875082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.875775 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:54.876865 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.877661 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:54.879368 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:54.879839 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:54.880206 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:54.880370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.880935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:54.880935 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:54.880935 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:54.880935 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:54.880935 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:54.880935 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:54.880935 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.880935 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:54.882291 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:54.882905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:54.882905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:54.882905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:54.882905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:54.882905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:54.882905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:54.882905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:54.882905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.882905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:54.883987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:54.884512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:54.885024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:54.885024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:54.885024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:55.002847 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:55.003065 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:55.003530 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:55.003780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:55.004380 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:50:55.004380 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:55.004380 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:55.004380 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:55.004380 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:55.004380 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:55.004380 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:55.004380 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:55.005075 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:55.005920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:55.005920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:55.005920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:55.005920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:55.005920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:55.005920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:55.005920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:55.005920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:50:55.006306 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:55.007052 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:55.007901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:55.007901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:55.086955 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:55.087653 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:55.088147 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:55.088605 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:55.088791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:55.089905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:50:55.089905 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:55.089905 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:55.089905 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:55.089905 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:55.089905 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:55.089905 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:55.089905 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:55.092080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:55.092897 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:59.775714 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:59.777183 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:50:59.808569 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:59.808714 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.809146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:50:59.809146 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:59.809146 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:59.809146 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:59.809146 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:59.809146 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:59.809146 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.809146 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:59.810127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.810900 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:59.874269 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:59.875434 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:59.875721 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:59.875923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.876858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:50:59.876858 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:59.876858 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:59.876858 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:59.876858 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:59.876858 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:59.876858 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.876858 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:50:59.878614 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.879488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:50:59.880409 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:50:59.880466 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:50:59.881080 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:50:59.881080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.881840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:50:59.881840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:59.881840 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:50:59.881840 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:50:59.881840 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:59.881840 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:50:59.881840 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.881840 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:50:59.883455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:50:59.884108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:59.884108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:50:59.884108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:59.884108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:50:59.884108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:50:59.884108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:59.884108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:50:59.884108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.884108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:50:59.885080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:50:59.885533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:50:59.886025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:50:59.886025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:50:59.886025 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:00.003379 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:00.003712 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:00.004243 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:00.004669 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:00.005339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:00.005339 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:00.005339 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:00.005339 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:00.005339 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:00.005339 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:00.005339 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:00.005339 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:00.005974 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:00.006760 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:00.007472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:00.008347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:00.008347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:00.008347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:00.008347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:00.008347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:00.008347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:00.008347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:00.008347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:00.008347 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:00.087499 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:00.087968 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:00.088463 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:00.088800 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:00.089015 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:00.089812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:51:00.089812 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:00.089812 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:00.089812 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:00.089812 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:00.089812 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:00.089812 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:00.089812 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:00.091462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:00.092313 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:04.776471 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:04.778141 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:04.812817 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:04.813252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.814097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:04.814097 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:04.814097 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:04.814097 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:04.814097 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:04.814097 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:04.814097 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.814097 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:04.814994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.815885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:04.874678 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:04.875755 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:04.876072 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:04.876361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.877358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:04.877358 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:04.877358 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:04.877358 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:04.877358 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:04.877358 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:04.877358 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.877358 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:04.879132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.879981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:04.880884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:04.880884 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:04.881450 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:04.881683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.882526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:51:04.882526 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:04.882526 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:04.882526 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:04.882526 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:04.882526 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:04.882526 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.882526 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:04.883847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:04.884356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:04.884356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:04.884356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:04.884356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:04.884356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:04.884356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:04.884356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:04.884356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.884356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:04.885566 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:04.886036 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:04.886569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:04.886569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:04.886569 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:05.003959 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:05.004457 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:05.005026 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:05.005455 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:05.006256 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:05.006256 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:05.006256 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:05.006256 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:05.006256 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:05.006256 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:05.006256 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:05.006256 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:05.007042 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:05.007764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:05.007764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:05.007764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:05.007764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:05.007764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:05.007764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:05.007764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:05.007764 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:05.008130 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:05.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:05.009763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:05.009763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:05.087931 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:05.088469 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:05.088942 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:05.089265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:05.089468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:05.090259 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:51:05.090259 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:05.090259 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:05.090259 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:05.090259 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:05.090259 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:05.090259 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:05.090259 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:05.091876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:05.092687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:09.777197 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:09.778702 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:09.815325 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:09.815790 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.816736 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:09.816736 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:09.816736 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:09.816736 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:09.816736 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:09.816736 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:09.816736 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.816736 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:09.817628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.818543 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:09.875456 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:09.876919 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:09.877226 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:09.877534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.878577 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:09.878577 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:09.878577 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:09.878577 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:09.878577 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:09.878577 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:09.878577 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.878577 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:09.880393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.881121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:09.881935 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:09.881935 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:09.881935 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:09.881935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.882351 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:51:09.882351 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:09.882351 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:09.882351 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:09.882351 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:09.882351 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:09.882351 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.882351 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:09.883757 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:09.884544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:09.884544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:09.884544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:09.884544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:09.884544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:09.884544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:09.884544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:09.884544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.884544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:09.885396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:09.886108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:09.886740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:09.886740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:09.886740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:10.004623 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:10.005454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:10.005999 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:10.006511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:10.007652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:10.007652 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:10.007652 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:10.007652 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:10.007652 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:10.007652 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:10.007652 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:10.007652 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:10.008912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:10.009531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:10.009531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:10.009531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:10.009531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:10.009531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:10.009531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:10.009531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:10.009531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:10.010553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:10.011563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:10.012542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:10.012542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:10.088575 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:10.090206 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:10.090911 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:10.091495 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:10.092381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:10.094687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:51:10.094687 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:10.094687 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:10.094687 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:10.094687 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:10.094687 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:10.094687 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:10.094687 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:10.095285 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:10.096318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:14.778120 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:14.779818 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:14.813632 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:14.813960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.814765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:14.814765 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:14.814765 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:14.814765 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:14.814765 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:14.814765 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:14.814765 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.814765 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:14.815834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.816441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:14.876635 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:14.877738 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:14.878040 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:14.878328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.879176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:14.879176 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:14.879176 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:14.879176 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:14.879176 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:14.879176 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:14.879176 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.879176 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:14.880491 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.880946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:14.881462 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:14.881914 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:14.882535 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:14.882805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.883667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:51:14.883667 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:14.883667 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:14.883667 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:14.883667 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:14.883667 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:14.883667 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.883667 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:14.884994 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:14.885512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:14.885512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:14.885512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:14.885512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:14.885512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:14.885512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:14.885512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:14.885512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.885512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:14.886631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:14.887097 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:14.887601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:14.887601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:14.887601 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:15.005313 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:15.005842 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:15.006459 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:15.006850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:15.007554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:15.007554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:15.007554 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:15.007554 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:15.007554 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:15.007554 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:15.007554 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:15.007554 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:15.008459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:15.009277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:15.009277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:15.009277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:15.009277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:15.009277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:15.009277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:15.009277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:15.009277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:15.009678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:15.010513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:15.011505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:15.011505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:15.089350 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:15.089936 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:15.090548 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:15.090889 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:15.091106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:15.091885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Traceback (most recent call last):
Mar 23 22:51:15.091885 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:15.091885 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:15.091885 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:15.091885 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:15.091885 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:15.091885 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:15.091885 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-af754a7a-a209-4e47-a7d4-48a6d48791c8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:15.093684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:15.094525 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:19.778938 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:19.780816 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:19.812973 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:19.813708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.814202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:19.814202 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:19.814202 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:19.814202 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:19.814202 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:19.814202 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:19.814202 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.814202 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:19.815417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.815938 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:19.877412 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:19.878629 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:19.878953 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:19.879192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.880429 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:19.880429 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:19.880429 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:19.880429 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:19.880429 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:19.880429 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:19.880429 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.880429 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:19.882597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.883339 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:19.884172 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:19.884172 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:19.884172 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:19.884427 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.884880 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:51:19.884880 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:19.884880 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:19.884880 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:19.884880 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:19.884880 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:19.884880 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.884880 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:19.886311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:19.886783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:19.886783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:19.886783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:19.886783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:19.886783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:19.886783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:19.886783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:19.886783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.886783 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:19.888048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:19.888534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:19.889011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:19.889011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:19.889011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:20.005817 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:20.006715 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:20.007285 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:20.007638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:20.008415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:20.008415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:20.008415 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:20.008415 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:20.008415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:20.008415 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:20.008415 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:20.008415 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:20.009836 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:20.010180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:20.010180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:20.010180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:20.010180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:20.010180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:20.010180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:20.010180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:20.010180 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:20.011494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:20.011849 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:20.012250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:20.012250 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:20.089906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:20.090574 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:20.091099 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:20.091555 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:20.091797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:20.092762 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:51:20.092762 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:20.092762 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:20.092762 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:20.092762 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:20.092762 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:20.092762 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:20.092762 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:20.094721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:20.095655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:24.779728 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:24.781343 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:24.812105 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:24.812105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.813263 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:24.813263 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:24.813263 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:24.813263 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:24.813263 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:24.813263 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:24.813263 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.813263 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:24.813802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.814653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:24.878073 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:24.879363 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:24.879733 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:24.879917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.880982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:24.880982 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:24.880982 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:24.880982 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:24.880982 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:24.880982 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:24.880982 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.880982 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:24.883333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.884436 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:24.885797 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:24.885797 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:24.885797 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:24.885797 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.886370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:51:24.886370 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:24.886370 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:24.886370 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:24.886370 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:24.886370 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:24.886370 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.886370 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:24.887759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:24.888810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:24.888810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:24.888810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:24.888810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:24.888810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:24.888810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:24.888810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:24.888810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.888810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:24.889407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:24.890349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:24.891089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:24.891089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:24.891089 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:25.007117 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:25.007425 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:25.008097 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:25.008565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:25.009346 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:25.009346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:25.009346 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:25.009346 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:25.009346 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:25.009346 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:25.009346 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:25.009346 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:25.010336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:25.011393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:25.011393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:25.011393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:25.011393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:25.011393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:25.011393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:25.011393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:25.011393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:25.011804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:25.012245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:25.012704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:25.012704 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:25.090560 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:25.091160 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:25.091843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:25.092264 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:25.092536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:25.093357 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:51:25.093357 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:25.093357 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:25.093357 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:25.093357 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:25.093357 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:25.093357 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:25.093357 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:25.094606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:25.095445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:29.780592 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:29.781943 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:29.822756 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:29.823364 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.824385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:29.824385 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:29.824385 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:29.824385 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:29.824385 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:29.824385 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:29.824385 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.824385 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:29.825668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.826499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:29.878893 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:29.880260 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:29.880596 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:29.880835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.881850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:29.881850 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:29.881850 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:29.881850 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:29.881850 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:29.881850 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:29.881850 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.881850 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:29.883604 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.884104 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:29.884834 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:29.884834 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:29.885294 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:29.885590 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.886377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:51:29.886377 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:29.886377 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:29.886377 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:29.886377 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:29.886377 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:29.886377 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.886377 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:29.887851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:29.888322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:29.888322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:29.888322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:29.888322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:29.888322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:29.888322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:29.888322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:29.888322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.888322 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:29.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:29.890048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:29.890589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:29.890589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:29.890589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:30.007654 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:30.008098 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:30.008653 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:30.009007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:30.009753 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:30.009753 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:30.009753 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:30.009753 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:30.009753 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:30.009753 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:30.009753 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:30.009753 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:30.010872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:30.011374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:30.011374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:30.011374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:30.011374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:30.011374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:30.011374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:30.011374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:30.011374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:30.012533 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:30.012982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:30.013500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:30.013500 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:30.091231 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:30.092047 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:30.092640 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:30.093030 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:30.093266 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:30.094145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:51:30.094145 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:30.094145 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:30.094145 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:30.094145 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:30.094145 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:30.094145 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:30.094145 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:30.096662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:30.097796 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:34.781183 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:34.783310 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:34.785335 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:51:34.824628 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:34.824818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.825452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:34.825452 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:34.825452 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:34.825452 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:34.825452 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:34.825452 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:34.825452 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.825452 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:34.826539 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:34.879552 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:34.880653 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:34.880874 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:34.881016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.881833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:34.881833 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:34.881833 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:34.881833 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:34.881833 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:34.881833 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:34.881833 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.881833 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:34.883715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.884524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:34.885376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:34.885376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:34.885524 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:34.885631 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.886243 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:51:34.886243 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:34.886243 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:34.886243 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:34.886243 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:34.886243 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:34.886243 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.886243 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:34.887556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:34.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:34.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:34.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:34.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:34.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:34.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:34.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:34.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.888487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:34.889032 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:34.889512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:34.890029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:34.890029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:34.890029 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:35.008288 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:35.008755 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:35.009448 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:35.009746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:35.010396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:35.010396 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:35.010396 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:35.010396 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:35.010396 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:35.010396 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:35.010396 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:35.010396 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:35.011729 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:35.012517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:35.012517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:35.012517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:35.012517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:35.012517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:35.012517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:35.012517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:35.012517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:35.013296 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:35.014044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:35.015157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:35.015157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:35.092089 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:35.092701 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:35.093100 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:35.093364 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:35.093571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:35.094137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:51:35.094137 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:35.094137 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:35.094137 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:35.094137 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:35.094137 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:35.094137 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:35.094137 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:35.095949 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:35.096917 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:39.782041 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:39.783856 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:39.822612 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:39.823043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.824002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:39.824002 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:39.824002 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:39.824002 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:39.824002 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:39.824002 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:39.824002 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.824002 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:39.824791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.825857 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:39.880051 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:39.881186 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:39.881437 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:39.881437 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.881882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:39.881882 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:39.881882 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:39.881882 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:39.881882 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:39.881882 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:39.881882 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.881882 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:39.882751 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.883261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:39.885070 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:39.885786 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:39.886153 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:39.886321 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.887567 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:51:39.887567 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:39.887567 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:39.887567 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:39.887567 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:39.887567 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:39.887567 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.887567 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:39.888166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:39.888747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:39.888747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:39.888747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:39.888747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:39.888747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:39.888747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:39.888747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:39.888747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.888747 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:39.889397 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:39.889889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:39.890481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:39.890481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:39.890481 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:40.008888 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:40.009350 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:40.009921 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:40.010388 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:40.011176 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:40.011176 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:40.011176 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:40.011176 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:40.011176 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:40.011176 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:40.011176 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:40.011176 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:40.012652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:40.013396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:40.013396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:40.013396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:40.013396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:40.013396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:40.013396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:40.013396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:40.013396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:40.014230 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:40.015000 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:40.015792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:40.015792 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:40.092642 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:40.093243 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:40.093912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:40.094367 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:40.094623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:40.095514 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:51:40.095514 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:40.095514 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:40.095514 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:40.095514 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:40.095514 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:40.095514 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:40.095514 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:40.097107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:40.097918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:44.782765 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:44.810715 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:44.839732 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:44.839874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.840340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:44.840340 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:44.840340 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:44.840340 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:44.840340 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:44.840340 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:44.840340 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.840340 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:44.841090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.841659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:44.880631 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:44.881804 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:44.882118 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:44.882419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.883396 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:44.883396 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:44.883396 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:44.883396 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:44.883396 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:44.883396 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:44.883396 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.883396 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:44.885171 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.885657 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:44.886177 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:44.886177 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:44.886749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:44.887023 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.887758 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:51:44.887758 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:44.887758 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:44.887758 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:44.887758 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:44.887758 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:44.887758 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.887758 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:44.889345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:44.889842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:44.889842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:44.889842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:44.889842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:44.889842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:44.889842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:44.889842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:44.889842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.889842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:44.891011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:44.891576 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:44.892105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:44.892105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:44.892105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:45.009396 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:45.009965 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:45.010667 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:45.011049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:45.011958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:45.011958 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:45.011958 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:45.011958 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:45.011958 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:45.011958 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:45.011958 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:45.011958 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:45.013330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:45.014002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:45.014002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:45.014002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:45.014002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:45.014002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:45.014002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:45.014002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:45.014002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:45.015385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:45.016142 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:45.016957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:45.016957 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:45.093134 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:45.093638 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:45.094245 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:45.094686 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:45.094981 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:45.095991 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:51:45.095991 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:45.095991 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:45.095991 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:45.095991 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:45.095991 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:45.095991 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:45.095991 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:45.097987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:45.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:49.783476 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:49.785063 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:49.821096 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:49.821363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.821920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:49.821920 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:49.821920 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:49.821920 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:49.821920 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:49.821920 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:49.821920 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.821920 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:49.822968 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.823808 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:49.881242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:49.882380 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:49.882545 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:49.882692 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.883320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:49.883320 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:49.883320 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:49.883320 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:49.883320 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:49.883320 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:49.883320 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.883320 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:49.884706 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.885454 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:49.885943 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:49.886449 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:49.886766 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:49.886936 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.887478 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:51:49.887478 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:49.887478 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:49.887478 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:49.887478 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:49.887478 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:49.887478 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.887478 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:49.888786 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:49.889255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:49.889255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:49.889255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:49.889255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:49.889255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:49.889255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:49.889255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:49.889255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.889255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:49.890485 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:49.890920 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:49.891415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:49.891415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:49.891415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:50.010117 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:50.010757 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:50.011510 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:50.011878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:50.012620 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:50.012620 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:50.012620 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:50.012620 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:50.012620 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:50.012620 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:50.012620 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:50.012620 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:50.013943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:50.015098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:50.015098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:50.015098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:50.015098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:50.015098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:50.015098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:50.015098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:50.015098 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:50.015584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:50.016625 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:50.017685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:50.017685 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:50.093641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:50.094012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:50.094550 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:50.094888 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:50.095110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:50.095895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:51:50.095895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:50.095895 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:50.095895 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:50.095895 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:50.095895 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:50.095895 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:50.095895 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:50.097734 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:50.098537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:54.784140 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:54.785395 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:54.811867 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:54.812084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.812650 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:54.812650 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:54.812650 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:54.812650 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:54.812650 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:54.812650 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:54.812650 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.812650 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:54.813572 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.814356 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:54.881974 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:54.882968 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:54.883135 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:54.883264 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.883925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:54.883925 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:54.883925 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:54.883925 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:54.883925 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:54.883925 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:54.883925 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.883925 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:54.885767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.886530 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:54.887358 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:54.887358 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:54.887358 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:54.887358 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.887842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:51:54.887842 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:54.887842 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:54.887842 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:54.887842 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:54.887842 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:54.887842 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.887842 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:54.889189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:54.889887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:54.889887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:54.889887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:54.889887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:54.889887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:54.889887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:54.889887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:54.889887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.889887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:54.890782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:54.891265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:54.891800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:54.891800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:54.891800 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:55.010851 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:55.011457 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:55.012027 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:55.012231 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:55.013013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:51:55.013013 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:55.013013 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:55.013013 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:55.013013 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:55.013013 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:55.013013 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:55.013013 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:55.014461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:55.015157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:55.015157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:55.015157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:55.015157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:55.015157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:55.015157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:55.015157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:55.015157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:51:55.016410 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:55.017124 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:55.017961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:55.017961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:55.094285 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:55.094959 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:55.095580 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:55.095945 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:55.096179 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:55.097045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:51:55.097045 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:55.097045 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:55.097045 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:55.097045 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:55.097045 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:55.097045 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:55.097045 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:55.098820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:55.099791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:59.784852 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:59.786584 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:51:59.818485 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:59.818882 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.819501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:51:59.819501 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:59.819501 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:59.819501 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:59.819501 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:59.819501 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:59.819501 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.819501 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:59.820670 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.821947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:59.882595 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:59.883932 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:59.884148 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:59.884350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.885292 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:59.885292 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:59.885292 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:59.885292 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:59.885292 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:59.885292 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:59.885292 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.885292 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:51:59.887261 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.887971 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:51:59.888789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:51:59.888789 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:51:59.888923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:51:59.889175 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.889946 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:51:59.889946 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:59.889946 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:51:59.889946 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:51:59.889946 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:59.889946 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:51:59.889946 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.889946 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:51:59.891507 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:51:59.892510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:59.892510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:51:59.892510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:59.892510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:51:59.892510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:51:59.892510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:59.892510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:51:59.892510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.892510 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:51:59.893005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:51:59.893812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:51:59.894761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:51:59.894761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:51:59.894761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:00.011452 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:00.012007 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:00.012797 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:00.013151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:00.013960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:00.013960 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:00.013960 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:00.013960 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:00.013960 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:00.013960 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:00.013960 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:00.013960 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:00.015159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:00.015993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:00.015993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:00.015993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:00.015993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:00.015993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:00.015993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:00.015993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:00.015993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:00.016756 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:00.017626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:00.018664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:00.018664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:00.094866 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:00.095506 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:00.096280 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:00.096695 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:00.096934 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:00.097905 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:52:00.097905 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:00.097905 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:00.097905 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:00.097905 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:00.097905 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:00.097905 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:00.097905 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:00.099606 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:00.100469 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:04.785723 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:04.787502 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:04.819708 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:04.819970 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.820749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:04.820749 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:04.820749 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:04.820749 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:04.820749 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:04.820749 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:04.820749 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.820749 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:04.821524 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.822866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:04.883325 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:04.884443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:04.884739 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:04.884977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.885815 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:04.885815 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:04.885815 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:04.885815 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:04.885815 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:04.885815 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:04.885815 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.885815 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:04.887841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.888367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:04.888905 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:04.889012 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:04.889637 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:04.889901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.890805 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:04.890805 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:04.890805 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:04.890805 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:04.890805 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:04.890805 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:04.890805 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.890805 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:04.892418 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:04.892898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:04.892898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:04.892898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:04.892898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:04.892898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:04.892898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:04.892898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:04.892898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.892898 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:04.894152 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:04.894802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:04.895369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:04.895369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:04.895369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:05.012064 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:05.012515 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:05.013444 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:05.013444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:05.013929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:05.013929 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:05.013929 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:05.013929 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:05.013929 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:05.013929 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:05.013929 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:05.013929 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:05.015081 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:05.015597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:05.015597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:05.015597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:05.015597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:05.015597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:05.015597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:05.015597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:05.015597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:05.016682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:05.017166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:05.017690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:05.017690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:05.095377 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:05.096169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:05.096765 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:05.097105 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:05.097370 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:05.098195 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:52:05.098195 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:05.098195 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:05.098195 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:05.098195 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:05.098195 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:05.098195 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:05.098195 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:05.100151 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:05.100993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:09.786482 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:09.788251 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:09.824408 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:09.824810 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.825759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:09.825759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:09.825759 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:09.825759 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:09.825759 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:09.825759 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:09.825759 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.825759 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:09.826811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.827948 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:09.883988 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:09.885420 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:09.885860 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:09.886183 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.887372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:09.887372 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:09.887372 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:09.887372 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:09.887372 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:09.887372 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:09.887372 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.887372 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:09.889186 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.890031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:09.890937 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:09.890937 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:09.890937 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:09.890937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.891209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:09.891209 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:09.891209 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:09.891209 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:09.891209 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:09.891209 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:09.891209 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.891209 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:09.891977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:09.892788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:09.892788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:09.892788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:09.892788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:09.892788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:09.892788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:09.892788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:09.892788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.892788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:09.893334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:09.894115 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:09.895118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:09.895118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:09.895118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:10.012579 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:10.013134 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:10.013794 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:10.014099 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:10.014754 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:10.014754 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:10.014754 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:10.014754 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:10.014754 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:10.014754 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:10.014754 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:10.014754 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:10.016028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:10.016834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:10.016834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:10.016834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:10.016834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:10.016834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:10.016834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:10.016834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:10.016834 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:10.017730 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:10.018583 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:10.019438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:10.019438 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:10.096068 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:10.096802 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:10.097492 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:10.097977 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:10.098232 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:10.099090 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:52:10.099090 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:10.099090 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:10.099090 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:10.099090 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:10.099090 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:10.099090 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:10.099090 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:10.100899 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:10.101674 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:14.787163 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:14.789273 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:14.822993 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:14.822993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.824106 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:14.824106 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:14.824106 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:14.824106 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:14.824106 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:14.824106 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:14.824106 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.824106 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:14.825070 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.826160 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:14.884649 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:14.885700 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:14.886116 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:14.886394 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.887244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:14.887244 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:14.887244 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:14.887244 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:14.887244 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:14.887244 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:14.887244 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.887244 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:14.889476 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.890329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:14.891173 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:14.891173 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:14.892001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:14.892472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.893667 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:14.893667 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:14.893667 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:14.893667 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:14.893667 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:14.893667 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:14.893667 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.893667 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:14.894526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:14.895062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:14.895062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:14.895062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:14.895062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:14.895062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:14.895062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:14.895062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:14.895062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.895062 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:14.895594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:14.896311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:14.896806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:14.896806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:14.896806 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:15.013273 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:15.013957 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:15.014643 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:15.015013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:15.015771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:15.015771 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:15.015771 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:15.015771 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:15.015771 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:15.015771 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:15.015771 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:15.015771 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:15.017405 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:15.018588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:15.018588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:15.018588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:15.018588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:15.018588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:15.018588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:15.018588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:15.018588 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:15.019648 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:15.020626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:15.021563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:15.021563 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:15.096639 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:15.097096 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96b50231-d848-437c-9393-29d871719f74 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:15.098315 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:15.098801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:15.099049 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:15.100121 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96b50231-d848-437c-9393-29d871719f74 None None] Traceback (most recent call last):
Mar 23 22:52:15.100121 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:15.100121 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:15.100121 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:15.100121 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:15.100121 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:15.100121 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:15.100121 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-96b50231-d848-437c-9393-29d871719f74 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:15.101811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:15.102681 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:19.787888 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:19.789967 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:19.826559 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:19.827033 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.828272 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:19.828272 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:19.828272 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:19.828272 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:19.828272 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:19.828272 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:19.828272 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.828272 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:19.829621 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.830164 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:19.885348 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:19.886773 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:19.887173 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:19.887511 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.888557 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:19.888557 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:19.888557 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:19.888557 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:19.888557 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:19.888557 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:19.888557 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.888557 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:19.890518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.891155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:19.891697 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:19.891697 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:19.891792 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:19.892086 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.892958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:19.892958 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:19.892958 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:19.892958 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:19.892958 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:19.892958 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:19.892958 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.892958 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:19.893928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:19.894441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:19.894441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:19.894441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:19.894441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:19.894441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:19.894441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:19.894441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:19.894441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.894441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:19.894913 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:19.895399 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:19.895871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:19.895871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:19.895871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:20.013767 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:20.014470 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:20.015841 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:20.016420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:20.017537 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:20.017537 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:20.017537 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:20.017537 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:20.017537 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:20.017537 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:20.017537 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:20.017537 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:20.019154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:20.019958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:20.019958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:20.019958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:20.019958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:20.019958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:20.019958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:20.019958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:20.019958 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:20.021324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:20.022072 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:20.022972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:20.022972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:20.097520 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:20.098798 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:20.098798 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:20.099069 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:20.099512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:20.100207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:20.100207 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:20.100207 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:20.100207 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:20.100207 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:20.100207 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:20.100207 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:20.100207 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:20.101423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:20.102428 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:24.788649 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:24.790300 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:24.824101 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:24.824387 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.825083 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:24.825083 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:24.825083 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:24.825083 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:24.825083 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:24.825083 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:24.825083 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.825083 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:24.826316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.827366 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:24.885993 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:24.887169 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:24.887537 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:24.887782 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.888835 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:24.888835 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:24.888835 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:24.888835 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:24.888835 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:24.888835 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:24.888835 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.888835 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:24.890816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.891597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:24.892432 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:24.892432 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:24.892432 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:24.892628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.893153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:24.893153 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:24.893153 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:24.893153 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:24.893153 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:24.893153 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:24.893153 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.893153 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:24.894535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:24.895206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:24.895206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:24.895206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:24.895206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:24.895206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:24.895206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:24.895206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:24.895206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.895206 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:24.896076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:24.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:24.897119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:24.897119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:24.897119 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:25.014694 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:25.014970 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:25.015713 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:25.016140 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:25.016980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:25.016980 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:25.016980 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:25.016980 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:25.016980 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:25.016980 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:25.016980 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:25.016980 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:25.018295 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:25.019129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:25.019129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:25.019129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:25.019129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:25.019129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:25.019129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:25.019129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:25.019129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:25.019879 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:25.020700 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:25.021759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:25.021759 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:25.097877 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:25.098519 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:25.099050 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:25.099512 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:25.099741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:25.100536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:25.100536 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:25.100536 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:25.100536 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:25.100536 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:25.100536 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:25.100536 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:25.100536 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:25.102379 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:25.103133 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:29.789571 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:29.791303 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:29.824719 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:29.825184 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.826376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:29.826376 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:29.826376 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:29.826376 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:29.826376 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:29.826376 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:29.826376 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.826376 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:29.827721 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.828267 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:29.886834 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:29.888593 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:29.888938 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:29.889251 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.890205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:29.890205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:29.890205 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:29.890205 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:29.890205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:29.890205 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:29.890205 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.890205 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:29.892257 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.893162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:29.894050 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:29.894050 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:29.894050 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:29.894290 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.895159 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:29.895159 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:29.895159 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:29.895159 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:29.895159 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:29.895159 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:29.895159 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.895159 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:29.896635 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:29.897131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:29.897131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:29.897131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:29.897131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:29.897131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:29.897131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:29.897131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:29.897131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.897131 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:29.898472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:29.899035 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:29.899571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:29.899571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:29.899571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:30.015367 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:30.016070 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:30.016746 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:30.017118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:30.017933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:30.017933 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:30.017933 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:30.017933 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:30.017933 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:30.017933 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:30.017933 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:30.017933 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:30.019385 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:30.020147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:30.020147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:30.020147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:30.020147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:30.020147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:30.020147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:30.020147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:30.020147 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:30.021509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:30.022452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:30.023374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:30.023374 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:30.098494 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:30.099244 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:30.099686 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:30.100042 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:30.100294 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:30.101415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:30.101415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:30.101415 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:30.101415 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:30.101415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:30.101415 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:30.101415 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:30.101415 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:30.103013 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:30.103896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:34.790172 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:34.792438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:34.794671 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=240 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:52:34.822760 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:34.823027 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.823851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:34.823851 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:34.823851 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:34.823851 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:34.823851 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:34.823851 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:34.823851 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.823851 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:34.824690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.825910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:34.887902 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:34.889622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:34.889870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:34.890157 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.891249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:34.891249 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:34.891249 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:34.891249 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:34.891249 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:34.891249 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:34.891249 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.891249 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:34.893248 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.894038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.895001 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:34.895683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:34.896487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:34.896487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:34.896487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:34.896487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:34.896487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:34.896487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:34.896487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:34.896487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.896487 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:34.897011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:34.897771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:34.898675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:34.898675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:34.898675 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:35.016248 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:35.016962 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:35.017687 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:35.018046 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:35.018842 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:35.018842 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:35.018842 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:35.018842 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:35.018842 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:35.018842 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:35.018842 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:35.018842 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:35.020156 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:35.020947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:35.020947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:35.020947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:35.020947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:35.020947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:35.020947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:35.020947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:35.020947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:35.021773 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:35.022597 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:35.023724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:35.023724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:35.099294 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:35.099534 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:35.099967 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:35.100416 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:35.100636 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:35.101327 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:35.101327 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:35.101327 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:35.101327 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:35.101327 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:35.101327 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:35.101327 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:35.101327 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:35.102445 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:35.103207 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:39.360199 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:39.372139 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:389}}
Mar 23 22:52:39.372385 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=104232) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:370}}
Mar 23 22:52:39.372571 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Fixing resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:52:39.407768 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, resource={'id': '06cffee1-b8de-433e-a67a-ec0e950c9238', 'name': 'private', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['130adef4-306b-4397-a096-91ee5094d0e2', 'fb1912da-e258-4d41-8cd4-7896f0de6385'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:17:25Z', 'updated_at': '2026-03-23T22:17:28Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 36968}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.408176 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-06cffee1-b8de-433e-a67a-ec0e950c9238, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.408603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c4286b-d53b-4217-a9f3-05177afa9297, external_ids_update=None, columns={'tag': 36968}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.408833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.409660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Traceback (most recent call last):
Mar 23 22:52:39.409660 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.409660 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:39.409660 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.409660 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.409660 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:39.409660 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.409660 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:39.410961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Failed to fix resource 06cffee1-b8de-433e-a67a-ec0e950c9238 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.410961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:52:39.410961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:52:39.410961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:52:39.410961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:52:39.410961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:52:39.410961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:52:39.410961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:39.410961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.410961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.410961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.411329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.411737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:52:39.411737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.411737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:52:39.411737 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Fixing resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:52:39.445617 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, resource={'id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'name': 'public', 'tenant_id': 'f1441a701b6342d6b11071933d14eb2c', 'project_id': 'f1441a701b6342d6b11071933d14eb2c', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'a53179c9-417a-4560-9bc7-e3f64551b15b'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-23T22:17:31Z', 'updated_at': '2026-03-23T22:17:40Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.446292 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.447170 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a8bd993e-d23e-4e73-8066-766307763bdb, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.447513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.448459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Traceback (most recent call last):
Mar 23 22:52:39.448459 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.448459 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:39.448459 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.448459 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.448459 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:39.448459 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.448459 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:39.449468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Failed to fix resource bb9d5c47-4694-42c2-a018-18e3c9058915 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.449468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:52:39.449468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:52:39.449468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:52:39.449468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:52:39.449468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:52:39.449468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:52:39.449468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:39.449468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.449468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.449468 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.450037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.450775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:52:39.450775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.450775 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:52:39.450775 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Fixing resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:52:39.482208 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, resource={'id': '3a042d54-7a79-465e-bd57-47cd6833a4d0', 'name': 'admin_net', 'tenant_id': 'b4ffb2af173e4612a30978ef510f4f44', 'project_id': 'b4ffb2af173e4612a30978ef510f4f44', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['935d20c1-7d6b-489a-8358-b5fb94945b7e'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:20:57Z', 'updated_at': '2026-03-23T22:20:58Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 47875}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.482664 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3a042d54-7a79-465e-bd57-47cd6833a4d0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.483178 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-36c442d0-efff-4c7e-bcb6-377b4da39c34, external_ids_update=None, columns={'tag': 47875}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.483402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.484297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Traceback (most recent call last):
Mar 23 22:52:39.484297 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.484297 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:39.484297 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.484297 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.484297 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:39.484297 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.484297 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:39.485360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Failed to fix resource 3a042d54-7a79-465e-bd57-47cd6833a4d0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.485360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:52:39.485360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:52:39.485360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:52:39.485360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:52:39.485360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:52:39.485360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:52:39.485360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:39.485360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.485360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.485360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.485901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.486592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:52:39.486592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.486592 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:52:39.486592 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Fixing resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:52:39.518689 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, resource={'id': 'c9e6c30a-b30e-473b-b06a-50be75c90b36', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['421749b7-bb0d-4854-b3f4-f0414f4c37fb'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-23T22:21:34Z', 'updated_at': '2026-03-23T22:21:35Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 48210}, resource_type=networks, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.519240 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c9e6c30a-b30e-473b-b06a-50be75c90b36, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.519832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13535482-a705-4453-af87-e8a7d8c10c82, external_ids_update=None, columns={'tag': 48210}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.520063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.521178 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Traceback (most recent call last):
Mar 23 22:52:39.521178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.521178 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:39.521178 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.521178 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.521178 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:39.521178 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.521178 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:39.522413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Failed to fix resource c9e6c30a-b30e-473b-b06a-50be75c90b36 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.522413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:52:39.522413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:52:39.522413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:52:39.522413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:52:39.522413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:52:39.522413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 23 22:52:39.522413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:39.522413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.522413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.522413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.522776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.523190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:52:39.523190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.523190 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:52:39.523190 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Fixing resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:52:39.604418 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, resource={'id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'project_id': 'a2ecc159a74349898a829ab82ef5cb4d', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'enable_snat': True}, 'gw_port_id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-t62km', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'external_fixed_ips': [{'ip_address': '172.24.5.146', 'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a'}, {'ip_address': '2001:db8::396', 'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 3, 'tenant_id': 'a2ecc159a74349898a829ab82ef5cb4d'}, resource_type=routers, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.604888 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-t62km', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.605265 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.605549 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-bb9d5c47-4694-42c2-a018-18e3c9058915, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "348aba7c-88a8-4a19-9486-df3f7edb4007"')], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.605763 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.606933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Traceback (most recent call last):
Mar 23 22:52:39.606933 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.606933 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:39.606933 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.606933 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.606933 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:39.606933 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.606933 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:39.607456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Unable to update router 154c08d6-78c5-49fd-a785-0df3f4277a23. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.608893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Failed to fix resource 154c08d6-78c5-49fd-a785-0df3f4277a23 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.608893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:52:39.608893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:52:39.608893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:52:39.608893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:52:39.608893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:52:39.608893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 23 22:52:39.608893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 23 22:52:39.608893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 23 22:52:39.608893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 23 22:52:39.608893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:52:39.609546 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:52:39.610058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:39.610058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:52:39.610058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.610058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:52:39.610058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.610058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.610058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:52:39.610058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.610058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:52:39.610058 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Fixing resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports) at create/update {{(pid=104232) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:399}}
Mar 23 22:52:39.720455 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, resource={'id': '348aba7c-88a8-4a19-9486-df3f7edb4007', 'name': '', 'network_id': 'bb9d5c47-4694-42c2-a018-18e3c9058915', 'project_id': '', 'mac_address': 'fa:16:3e:aa:0e:57', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '154c08d6-78c5-49fd-a785-0df3f4277a23', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a', 'ip_address': '172.24.5.146'}, {'subnet_id': 'a53179c9-417a-4560-9bc7-e3f64551b15b', 'ip_address': '2001:db8::396'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160382', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-23T22:21:36Z', 'updated_at': '2026-03-23T22:21:39Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.720886 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '97b5115a-76f2-40e9-9f0d-b5ab0f590b3a a53179c9-417a-4560-9bc7-e3f64551b15b', 'neutron:network_name': 'neutron-bb9d5c47-4694-42c2-a018-18e3c9058915', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-154c08d6-78c5-49fd-a785-0df3f4277a23'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::396/64', '172.24.5.146/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.721553 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=348aba7c-88a8-4a19-9486-df3f7edb4007, lrouter_port=lrp-348aba7c-88a8-4a19-9486-df3f7edb4007, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.722017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.723305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Traceback (most recent call last):
Mar 23 22:52:39.723305 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.723305 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:39.723305 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.723305 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.723305 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:39.723305 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.723305 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:39.724791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Failed to fix resource 348aba7c-88a8-4a19-9486-df3f7edb4007 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.724791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 23 22:52:39.724791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 412, in check_for_inconsistencies
Mar 23 22:52:39.724791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 23 22:52:39.724791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 321, in _fix_create_update
Mar 23 22:52:39.724791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 23 22:52:39.724791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 23 22:52:39.724791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:39.724791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.724791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.724791 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.725431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.726028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 23 22:52:39.726028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.726028 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 23 22:52:39.726028 np0000160382 neutron-ovn-maintenance-worker[104232]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d7b4c058-35ca-440b-a4d1-dabe8f7a4d8e None None] Maintenance task: Synchronization completed (took 0.35 seconds)
Mar 23 22:52:39.791013 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:39.792843 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:39.825954 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.826462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.827345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:39.827345 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.827345 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:39.827345 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.827345 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.827345 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:39.827345 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.827345 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.828316 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.829168 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:39.888538 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:39.889785 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.890291 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.890550 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.891654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:39.891654 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.891654 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:39.891654 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.891654 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.891654 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:39.891654 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.891654 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:39.893724 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.894522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:39.895352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:39.895352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:39.895352 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:39.895352 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.895966 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:39.895966 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.895966 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:39.895966 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.895966 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.895966 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:39.895966 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.895966 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:39.897237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:39.897964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:39.897964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:39.897964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.897964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:39.897964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.897964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.897964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:39.897964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.897964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:39.899008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:39.899887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:39.900840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:39.900840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:39.900840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:40.017583 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:40.018242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:40.019356 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:40.019804 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:40.020789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:40.020789 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:40.020789 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:40.020789 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:40.020789 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:40.020789 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:40.020789 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:40.020789 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:40.022336 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:40.022817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:40.022817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:40.022817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:40.022817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:40.022817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:40.022817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:40.022817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:40.022817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:40.024113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:40.024642 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:40.025132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:40.025132 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:40.099730 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:40.100177 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:40.100923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:40.101402 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:40.101659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:40.102923 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:40.102923 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:40.102923 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:40.102923 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:40.102923 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:40.102923 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:40.102923 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:40.102923 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:40.104664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:40.105470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:44.791740 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:44.793476 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:44.827120 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:44.827553 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.828549 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:44.828549 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:44.828549 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:44.828549 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:44.828549 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:44.828549 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:44.828549 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.828549 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:44.829814 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.830717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:44.889281 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:44.890454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:44.890836 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:44.891129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.892876 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:44.892876 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:44.892876 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:44.892876 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:44.892876 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:44.892876 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:44.892876 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.892876 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:44.896037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.896858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:44.897740 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:44.897740 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:44.897740 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:44.897935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.898654 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:44.898654 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:44.898654 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:44.898654 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:44.898654 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:44.898654 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:44.898654 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.898654 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:44.899663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:44.900591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:44.900591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:44.900591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:44.900591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:44.900591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:44.900591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:44.900591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:44.900591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.900591 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:44.901037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:44.901888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:44.902823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:44.902823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:44.902823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:45.018268 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:45.018722 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:45.019832 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:45.020360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:45.021717 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:45.021717 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:45.021717 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:45.021717 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:45.021717 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:45.021717 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:45.021717 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:45.021717 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:45.023369 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:45.024305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:45.024305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:45.024305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:45.024305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:45.024305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:45.024305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:45.024305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:45.024305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:45.025415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:45.026166 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:45.026947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:45.026947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:45.100374 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:45.100806 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:45.101453 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:45.101916 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:45.102155 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:45.103045 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:45.103045 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:45.103045 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:45.103045 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:45.103045 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:45.103045 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:45.103045 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:45.103045 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:45.104740 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:45.105581 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:49.792574 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:49.793880 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:49.827034 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:49.827462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.828381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:49.828381 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:49.828381 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:49.828381 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:49.828381 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:49.828381 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:49.828381 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.828381 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:49.829526 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.830433 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:49.890092 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:49.891360 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:49.891694 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:49.891942 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.892925 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:49.892925 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:49.892925 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:49.892925 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:49.892925 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:49.892925 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:49.892925 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.892925 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:49.895017 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.895535 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:49.896887 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:49.897561 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:49.897809 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:49.898002 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.898651 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:49.898651 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:49.898651 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:49.898651 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:49.898651 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:49.898651 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:49.898651 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.898651 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:49.899655 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:49.900542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:49.901372 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:49.902297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:50.018973 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:50.019595 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:50.020171 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:50.020618 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:50.021459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:50.021459 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:50.021459 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:50.021459 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:50.021459 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:50.021459 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:50.021459 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:50.021459 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:50.022807 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:50.023570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:50.023570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:50.023570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:50.023570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:50.023570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:50.023570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:50.023570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:50.023570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:50.024415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:50.025271 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:50.026170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:50.026170 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:50.101275 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:50.101793 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:50.102307 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:50.102716 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:50.103519 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:50.103794 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:50.103794 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:50.103794 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:50.103794 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:50.103794 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:50.103794 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:50.103794 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:50.103794 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:50.105420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:50.105953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:54.793389 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:54.794822 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:54.833762 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:54.833962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.834735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:54.834735 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:54.834735 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:54.834735 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:54.834735 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:54.834735 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:54.834735 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.834735 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:54.835866 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.836928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:54.890775 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:54.891906 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:54.892246 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:54.892536 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.893632 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:54.893632 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:54.893632 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:54.893632 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:54.893632 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:54.893632 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:54.893632 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.893632 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:54.894862 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.896112 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:54.897438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:54.897818 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:54.898393 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:54.898709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.899558 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:54.899558 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:54.899558 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:54.899558 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:54.899558 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:54.899558 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:54.899558 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.899558 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:54.900822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:54.901612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:54.901612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:54.901612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:54.901612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:54.901612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:54.901612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:54.901612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:54.901612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.901612 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:54.902541 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:54.903432 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:54.904348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:54.904348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:54.904348 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:55.019748 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:55.020918 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:55.022037 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:55.022522 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:55.023471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:55.023471 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:55.023471 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:55.023471 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:55.023471 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:55.023471 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:55.023471 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:55.023471 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:55.025363 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:55.026153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:55.026153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:55.026153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:55.026153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:55.026153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:55.026153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:55.026153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:55.026153 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:52:55.027137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:55.027907 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:55.028943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:55.028943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:55.102055 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:55.102808 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:55.103386 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:55.103756 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:55.103996 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:55.104850 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:52:55.104850 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:55.104850 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:55.104850 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:55.104850 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:55.104850 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:55.104850 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:55.104850 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:55.106962 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:55.107735 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:59.794090 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:59.796988 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:52:59.830063 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:59.830063 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.830659 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:52:59.830659 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:59.830659 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:59.830659 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:59.830659 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:59.830659 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:59.830659 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.830659 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:59.832107 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.832587 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:59.891483 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:59.892295 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:59.892523 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:59.892663 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.893365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:52:59.893365 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:59.893365 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:59.893365 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:59.893365 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:59.893365 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:59.893365 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.893365 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:52:59.895037 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.895947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:52:59.897801 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:52:59.898064 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:52:59.898401 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:52:59.898552 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.899415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:52:59.899415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:59.899415 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:52:59.899415 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:52:59.899415 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:59.899415 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:52:59.899415 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.899415 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:52:59.900839 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:52:59.901402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:59.901402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:52:59.901402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:59.901402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:52:59.901402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:52:59.901402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:59.901402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:52:59.901402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.901402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:52:59.902774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:52:59.903310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:52:59.903844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:52:59.903844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:52:59.903844 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:00.020581 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:00.021126 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:00.021572 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:00.021801 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:00.022354 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:00.022354 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:00.022354 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:00.022354 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:00.022354 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:00.022354 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:00.022354 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:00.022354 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:00.023877 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:00.024686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:00.024686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:00.024686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:00.024686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:00.024686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:00.024686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:00.024686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:00.024686 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:00.025240 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:00.026011 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:00.027016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:00.027016 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:00.102736 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:00.103188 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:00.103726 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:00.104085 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:00.104328 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:00.105044 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:00.105044 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:00.105044 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:00.105044 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:00.105044 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:00.105044 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:00.105044 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:00.105044 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:00.106239 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:00.106911 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:04.795666 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:04.797438 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:04.833548 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:04.833960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.834823 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:04.834823 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:04.834823 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:04.834823 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:04.834823 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:04.834823 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:04.834823 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.834823 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:04.835961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.836859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:04.891923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:04.893107 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:04.893474 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:04.893715 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.894726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:04.894726 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:04.894726 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:04.894726 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:04.894726 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:04.894726 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:04.894726 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.894726 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:04.896633 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.897392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:04.898200 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:04.898865 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:04.899418 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:04.899720 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.900477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:53:04.900477 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:04.900477 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:04.900477 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:04.900477 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:04.900477 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:04.900477 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.900477 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:04.901809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:04.902275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:04.902275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:04.902275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:04.902275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:04.902275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:04.902275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:04.902275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:04.902275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.902275 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:04.903470 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:04.903935 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:04.904440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:04.904440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:04.904440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:05.021152 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:05.021786 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:05.022415 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:05.022780 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:05.023599 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:05.023599 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:05.023599 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:05.023599 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:05.023599 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:05.023599 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:05.023599 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:05.023599 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:05.024929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:05.025713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:05.025713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:05.025713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:05.025713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:05.025713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:05.025713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:05.025713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:05.025713 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:05.026582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:05.027521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:05.028305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:05.028305 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:05.103421 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:05.103858 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:05.104321 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:05.104669 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:05.104889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:05.105586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:05.105586 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:05.105586 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:05.105586 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:05.105586 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:05.105586 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:05.105586 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:05.105586 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:05.106472 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:05.107413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:09.796562 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:09.798632 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:09.835092 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:09.835542 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.836825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:09.836825 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:09.836825 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:09.836825 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:09.836825 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:09.836825 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:09.836825 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.836825 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:09.838262 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.838995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:09.892657 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:09.893690 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:09.894033 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:09.894312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.895417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:09.895417 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:09.895417 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:09.895417 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:09.895417 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:09.895417 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:09.895417 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.895417 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:09.896646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.897338 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:09.898784 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:09.899304 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:09.899591 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:09.899789 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.900434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:53:09.900434 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:09.900434 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:09.900434 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:09.900434 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:09.900434 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:09.900434 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.900434 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:09.902117 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:09.902892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:09.902892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:09.902892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:09.902892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:09.902892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:09.902892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:09.902892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:09.902892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.902892 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:09.903341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:09.904020 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:09.904832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:09.904832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:09.904832 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:10.022162 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:10.022938 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:10.023714 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:10.024012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:10.024664 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:10.024664 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:10.024664 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:10.024664 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:10.024664 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:10.024664 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:10.024664 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:10.024664 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:10.026006 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:10.026937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:10.026937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:10.026937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:10.026937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:10.026937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:10.026937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:10.026937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:10.026937 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:10.027727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:10.028571 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:10.029492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:10.029492 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:10.104455 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:10.105073 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:10.105702 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:10.106077 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:10.106393 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:10.107192 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:10.107192 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:10.107192 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:10.107192 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:10.107192 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:10.107192 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:10.107192 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:10.107192 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:10.109494 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:10.110340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:14.797436 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:14.799300 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:14.834186 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:14.834641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.835497 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:14.835497 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:14.835497 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:14.835497 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:14.835497 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:14.835497 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:14.835497 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.835497 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:14.836365 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.836903 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:14.893261 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:14.894552 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:14.894870 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:14.895120 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.896141 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:14.896141 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:14.896141 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:14.896141 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:14.896141 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:14.896141 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:14.896141 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.896141 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:14.897972 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.898774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:14.899759 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:14.900592 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:14.900976 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:14.901167 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.902187 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:53:14.902187 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:14.902187 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:14.902187 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:14.902187 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:14.902187 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:14.902187 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.902187 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:14.904252 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:14.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:14.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:14.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:14.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:14.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:14.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:14.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:14.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:14.906039 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:14.906841 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:14.907883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:14.907883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:14.907883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:15.023013 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:15.023668 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:15.024376 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:15.024840 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:15.025825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:15.025825 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:15.025825 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:15.025825 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:15.025825 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:15.025825 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:15.025825 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:15.025825 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:15.027244 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:15.028149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:15.028149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:15.028149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:15.028149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:15.028149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:15.028149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:15.028149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:15.028149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:15.028825 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:15.029827 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:15.030802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:15.030802 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:15.105287 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:15.105713 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:15.106182 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:15.106589 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:15.106819 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:15.107628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:15.107628 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:15.107628 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:15.107628 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:15.107628 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:15.107628 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:15.107628 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:15.107628 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:15.109318 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:15.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:19.798242 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:19.799720 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:19.836260 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:19.836471 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.836918 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:19.836918 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:19.836918 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:19.836918 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:19.836918 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:19.836918 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:19.836918 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.836918 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:19.837977 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.838517 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:19.893865 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:19.894628 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:19.894808 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:19.895441 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.895874 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:19.895874 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:19.895874 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:19.895874 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:19.895874 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:19.895874 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:19.895874 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.895874 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:19.896749 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.897310 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:19.899759 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:19.900200 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:19.900812 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:19.901077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.902022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:53:19.902022 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:19.902022 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:19.902022 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:19.902022 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:19.902022 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:19.902022 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.902022 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:19.903162 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:19.903634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:19.903634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:19.903634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:19.903634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:19.903634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:19.903634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:19.903634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:19.903634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.903634 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:19.905003 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:19.905521 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:19.906038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:19.906038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:19.906038 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:20.023660 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:20.024166 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:20.024806 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:20.025074 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:20.025641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:20.025641 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:20.025641 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:20.025641 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:20.025641 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:20.025641 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:20.025641 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:20.025641 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:20.026444 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:20.026989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:20.026989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:20.026989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:20.026989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:20.026989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:20.026989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:20.026989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:20.026989 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:20.027417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:20.028069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:20.028767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:20.028767 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:20.105899 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:20.106389 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:20.106961 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:20.107159 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:20.107278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:20.107741 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:20.107741 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:20.107741 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:20.107741 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:20.107741 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:20.107741 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:20.107741 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:20.107741 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:20.108477 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:20.109486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:24.798750 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:24.800460 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:24.834559 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:24.834995 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.835910 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:24.835910 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:24.835910 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:24.835910 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:24.835910 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:24.835910 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:24.835910 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.835910 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:24.837082 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.837980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:24.894394 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:24.895561 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:24.895912 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:24.896177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.897349 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:24.897349 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:24.897349 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:24.897349 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:24.897349 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:24.897349 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:24.897349 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.897349 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:24.899312 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.900102 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:24.901058 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:24.901058 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:24.901652 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:24.902022 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.902838 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:53:24.902838 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:24.902838 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:24.902838 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:24.902838 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:24.902838 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:24.902838 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.902838 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:24.903726 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:24.904197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:24.904197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:24.904197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:24.904197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:24.904197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:24.904197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:24.904197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:24.904197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.904197 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:24.904931 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:24.905452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:24.905951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:24.905951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:24.905951 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:25.024533 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:25.025031 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:25.025674 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:25.026161 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:25.026988 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:25.026988 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:25.026988 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:25.026988 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:25.026988 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:25.026988 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:25.026988 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:25.026988 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:25.027885 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:25.028646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:25.028646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:25.028646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:25.028646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:25.028646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:25.028646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:25.028646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:25.028646 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:25.029056 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:25.029829 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:25.030746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:25.030746 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:25.106554 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:25.107163 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:25.107694 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:25.108110 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:25.108421 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:25.109333 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:25.109333 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:25.109333 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:25.109333 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:25.109333 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:25.109333 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:25.109333 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:25.109333 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:25.111154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:25.111901 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:29.799443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:29.802176 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:29.833579 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:29.833774 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.834464 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:29.834464 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:29.834464 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:29.834464 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:29.834464 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:29.834464 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:29.834464 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.834464 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:29.835461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.836528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:29.895088 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:29.896183 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:29.896587 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:29.896868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.897883 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:29.897883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:29.897883 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:29.897883 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:29.897883 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:29.897883 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:29.897883 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.897883 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:29.899076 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.899816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:29.900718 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:29.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:29.901974 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:29.902381 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.903335 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:53:29.903335 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:29.903335 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:29.903335 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:29.903335 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:29.903335 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:29.903335 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.903335 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:29.904710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:29.905341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:29.905341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:29.905341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:29.905341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:29.905341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:29.905341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:29.905341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:29.905341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.905341 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:29.905709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:29.906169 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:29.906719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:29.906719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:29.906719 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:30.025252 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:30.025958 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:30.026648 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:30.027091 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:30.028205 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:30.028205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:30.028205 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:30.028205 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:30.028205 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:30.028205 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:30.028205 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:30.028205 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:30.029627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:30.030423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:30.030423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:30.030423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:30.030423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:30.030423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:30.030423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:30.030423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:30.030423 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:30.031987 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:30.032859 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:30.033376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:30.033376 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:30.107380 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:30.107810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:30.108411 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:30.108876 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:30.109108 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:30.109896 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:30.109896 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:30.109896 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:30.109896 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:30.109896 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:30.109896 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:30.109896 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:30.109896 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:30.111738 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:30.112137 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:34.800255 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:34.801603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:34.803280 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG dbcounter [-] [104232] Writing DB stats neutron:SELECT=434 {{(pid=104232) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 23 22:53:34.833813 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:34.834064 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.835024 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:34.835024 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:34.835024 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:34.835024 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:34.835024 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:34.835024 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:34.835024 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.835024 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:34.835928 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.837269 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:34.895749 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:34.896890 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:34.897235 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:34.897509 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.898490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:34.898490 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:34.898490 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:34.898490 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:34.898490 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:34.898490 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:34.898490 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.898490 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:34.900334 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.901499 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:34.902709 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:34.902709 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:34.902709 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:34.902709 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.903055 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:53:34.903055 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:34.903055 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:34.903055 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:34.903055 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:34.903055 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:34.903055 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.903055 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:34.904311 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:34.905297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:34.905297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:34.905297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:34.905297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:34.905297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:34.905297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:34.905297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:34.905297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.905297 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:34.906031 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:34.906699 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:34.907490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:34.907490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:34.907490 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:35.025996 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:35.026651 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:35.027241 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:35.027673 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:35.028486 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:35.028486 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:35.028486 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:35.028486 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:35.028486 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:35.028486 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:35.028486 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:35.028486 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:35.029888 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:35.030653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:35.030653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:35.030653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:35.030653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:35.030653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:35.030653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:35.030653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:35.030653 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:35.032174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:35.033174 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:35.034145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:35.034145 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:35.107810 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:35.108267 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:35.108698 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:35.109137 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:35.109407 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:35.110134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:35.110134 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:35.110134 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:35.110134 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:35.110134 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:35.110134 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:35.110134 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:35.110134 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:35.111812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:35.112603 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:39.801126 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:39.802982 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:39.846104 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:39.846430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.847361 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:39.847361 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:39.847361 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:39.847361 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:39.847361 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:39.847361 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:39.847361 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.847361 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:39.848406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.849043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:39.896537 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:39.898367 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:39.898367 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:39.898690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.899727 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:39.899727 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:39.899727 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:39.899727 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:39.899727 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:39.899727 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:39.899727 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.899727 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:39.902118 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.902644 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:39.903158 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:39.903158 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:39.903312 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:39.903528 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.904069 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:39.904069 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:39.904069 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:39.904069 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:39.904069 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:39.904069 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:39.904069 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.904069 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:39.904947 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:39.905456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:39.905456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:39.905456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:39.905456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:39.905456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:39.905456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:39.905456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:39.905456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.905456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:39.905939 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:39.906417 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:39.906868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:39.906868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:39.906868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:40.026777 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:40.027534 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:40.027964 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:40.028503 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:40.029330 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:40.029330 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:40.029330 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:40.029330 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:40.029330 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:40.029330 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:40.029330 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:40.029330 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:40.030662 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:40.031431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:40.031431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:40.031431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:40.031431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:40.031431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:40.031431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:40.031431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:40.031431 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:40.032413 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:40.033462 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:40.034209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:40.034209 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:40.108614 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:40.109021 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:40.109506 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:40.109889 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:40.110127 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:40.110873 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:40.110873 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:40.110873 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:40.110873 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:40.110873 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:40.110873 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:40.110873 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:40.110873 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:40.112640 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:40.113513 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:44.801842 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:44.803156 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:44.835539 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:44.835811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.836582 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:44.836582 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:44.836582 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:44.836582 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:44.836582 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:44.836582 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:44.836582 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.836582 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:44.837415 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.838300 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:44.897311 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:44.898257 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:44.898546 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:44.898660 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.899430 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:44.899430 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:44.899430 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:44.899430 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:44.899430 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:44.899430 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:44.899430 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.899430 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:44.901377 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.902249 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:44.903049 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:44.903641 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:44.904050 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:44.904177 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.904737 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:53:44.904737 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:44.904737 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:44.904737 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:44.904737 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:44.904737 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:44.904737 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.904737 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:44.905638 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:44.906459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:44.906459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:44.906459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:44.906459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:44.906459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:44.906459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:44.906459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:44.906459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.906459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:44.906871 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:44.907610 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:44.908466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:44.908466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:44.908466 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:45.027671 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:45.028340 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:45.029104 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:45.029544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:45.030362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:45.030362 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:45.030362 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:45.030362 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:45.030362 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:45.030362 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:45.030362 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:45.030362 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:45.031281 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:45.031623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:45.031623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:45.031623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:45.031623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:45.031623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:45.031623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:45.031623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:45.031623 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:45.032406 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:45.032765 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:45.033116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:45.033116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:45.109387 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:45.109905 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:45.110510 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:45.110834 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:45.111149 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:45.111953 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:45.111953 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:45.111953 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:45.111953 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:45.111953 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:45.111953 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:45.111953 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:45.111953 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:45.113570 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:45.114761 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:49.802592 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:49.804317 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:49.836094 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:49.836452 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.837014 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:49.837014 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:49.837014 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:49.837014 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:49.837014 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:49.837014 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:49.837014 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.837014 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:49.837848 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.839345 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:49.897820 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:49.899072 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:49.899567 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:49.899771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.900818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:49.900818 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:49.900818 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:49.900818 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:49.900818 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:49.900818 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:49.900818 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.900818 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:49.902617 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.903671 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:49.904308 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:49.904308 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:49.904632 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:49.904872 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.905626 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:53:49.905626 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:49.905626 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:49.905626 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:49.905626 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:49.905626 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:49.905626 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.905626 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:49.907034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:49.907837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:49.907837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:49.907837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:49.907837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:49.907837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:49.907837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:49.907837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:49.907837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.907837 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:49.908668 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:49.909111 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:49.909818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:49.909818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:49.909818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:50.028622 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:50.028995 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:50.029752 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:50.030043 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:50.030683 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:50.030683 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:50.030683 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:50.030683 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:50.030683 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:50.030683 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:50.030683 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:50.030683 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:50.031956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:50.033122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:50.033122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:50.033122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:50.033122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:50.033122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:50.033122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:50.033122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:50.033122 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:50.033771 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:50.034809 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:50.036058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:50.036058 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:50.110035 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:50.110698 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:50.111384 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:50.111666 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:50.111833 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:50.112682 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:50.112682 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:50.112682 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:50.112682 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:50.112682 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:50.112682 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:50.112682 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:50.112682 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:50.113851 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:50.114652 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:54.803436 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:54.805419 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:54.840142 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:54.840584 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.841461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:54.841461 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:54.841461 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:54.841461 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:54.841461 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:54.841461 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:54.841461 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.841461 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:54.842505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.843512 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:54.898520 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:54.899652 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:54.899953 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:54.900194 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.901087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:54.901087 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:54.901087 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:54.901087 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:54.901087 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:54.901087 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:54.901087 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.901087 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:54.902237 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.903005 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:54.904058 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:54.904574 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:54.905023 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:54.905255 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.905788 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:53:54.905788 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:54.905788 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:54.905788 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:54.905788 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:54.905788 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:54.905788 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.905788 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:54.906708 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:54.907459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:54.907459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:54.907459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:54.907459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:54.907459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:54.907459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:54.907459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:54.907459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.907459 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:54.908146 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:54.908688 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:54.909278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:54.909278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:54.909278 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:55.029511 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:55.030054 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:55.030853 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:55.031340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:55.032173 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:55.032173 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:55.032173 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:55.032173 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:55.032173 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:55.032173 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:55.032173 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:55.032173 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:55.033501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:55.034265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:55.034265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:55.034265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:55.034265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:55.034265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:55.034265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:55.034265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:55.034265 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:53:55.035084 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:55.035904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:55.036847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:55.036847 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:55.110914 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:55.111371 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:55.111923 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:55.112393 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:55.112690 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:55.113473 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:53:55.113473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:55.113473 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:55.113473 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:55.113473 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:55.113473 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:55.113473 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:55.113473 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:55.114505 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:55.115479 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:59.804417 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:59.806007 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:53:59.840103 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:59.840531 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.841317 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:53:59.841317 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:59.841317 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:59.841317 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:59.841317 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:59.841317 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:59.841317 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.841317 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:59.842392 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.843402 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:59.899189 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:59.900385 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:59.900697 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:59.900933 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.901776 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:53:59.901776 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:59.901776 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:59.901776 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:59.901776 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:59.901776 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:59.901776 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.901776 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:53:59.903589 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.904451 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:53:59.905594 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:53:59.905594 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:53:59.905594 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:53:59.905594 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.906360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:53:59.906360 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:59.906360 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:53:59.906360 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:53:59.906360 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:59.906360 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:53:59.906360 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.906360 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:53:59.907518 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:53:59.908189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:59.908189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:53:59.908189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:59.908189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:53:59.908189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:53:59.908189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:59.908189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:53:59.908189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.908189 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:53:59.909034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:53:59.909565 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:53:59.910093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:53:59.910093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:53:59.910093 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:00.030118 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:00.030700 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:00.031341 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:00.031678 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:00.032434 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:54:00.032434 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:00.032434 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:00.032434 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:00.032434 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:00.032434 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:00.032434 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:00.032434 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:54:00.033766 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:00.034554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:54:00.034554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:00.034554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:54:00.034554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:54:00.034554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:00.034554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:54:00.034554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:00.034554 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:54:00.035360 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:00.036110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:00.037110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:00.037110 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:00.111387 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:00.111693 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:00.112302 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:00.112647 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:00.112868 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:00.113627 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:54:00.113627 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:00.113627 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:00.113627 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:00.113627 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:00.113627 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:00.113627 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:00.113627 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:00.115362 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:00.115943 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:04.805252 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:04.807053 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:04.841084 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:04.841480 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.842420 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:54:04.842420 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:04.842420 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:04.842420 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:04.842420 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:04.842420 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:04.842420 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.842420 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:04.843309 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.843912 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:04.900015 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:04.901276 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:04.901647 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:04.901884 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.902904 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:54:04.902904 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:04.902904 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:04.902904 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:04.902904 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:04.902904 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:04.902904 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.902904 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:04.904628 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.905600 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:04.906443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:04.906443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:54:04.906443 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:04.906443 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.906818 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:54:04.906818 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:04.906818 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:04.906818 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:04.906818 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:04.906818 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:04.906818 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.906818 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:54:04.908113 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:54:04.908641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:04.908641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:54:04.908641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:04.908641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:54:04.908641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:54:04.908641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:04.908641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:54:04.908641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.908641 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:54:04.909822 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:04.910324 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:04.910820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:04.910820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:04.910820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:05.030723 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:05.031577 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:05.032337 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:05.032950 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:05.033656 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:54:05.033656 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:05.033656 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:05.033656 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:05.033656 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:05.033656 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:05.033656 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:05.033656 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:54:05.035009 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:05.035544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:54:05.035544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:05.035544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:54:05.035544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:54:05.035544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:05.035544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:54:05.035544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:05.035544 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:54:05.036615 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:05.037080 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:05.037586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:05.037586 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:05.111796 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:05.112339 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:05.112857 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:05.113262 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:05.113501 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:05.114367 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:54:05.114367 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:05.114367 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:05.114367 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:05.114367 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:05.114367 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:05.114367 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:05.114367 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:05.116105 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:05.116926 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:09.806320 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:09.808426 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:09.840723 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:09.841116 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.842001 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:54:09.842001 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:09.842001 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:09.842001 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:09.842001 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:09.842001 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:09.842001 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.842001 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:09.842922 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.843893 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:09.900799 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:09.901826 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:09.902143 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:09.902419 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.903320 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:54:09.903320 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:09.903320 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:09.903320 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:09.903320 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:09.903320 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:09.903320 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.903320 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:09.905200 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.906077 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:09.906952 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:09.906952 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:54:09.906952 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:09.906952 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.908034 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:54:09.908034 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:09.908034 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:09.908034 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:09.908034 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:09.908034 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:09.908034 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.908034 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:54:09.908964 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:54:09.909508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:09.909508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:54:09.909508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:09.909508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:54:09.909508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:54:09.909508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:09.909508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:54:09.909508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.909508 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:54:09.910067 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:09.911087 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:09.911710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:09.911710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:09.911710 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:10.031503 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:10.032246 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:10.033312 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:10.033811 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:10.034816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:54:10.034816 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:10.034816 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:10.034816 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:10.034816 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:10.034816 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:10.034816 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:10.034816 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:54:10.036202 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:10.036743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:54:10.036743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:10.036743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:54:10.036743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:54:10.036743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:10.036743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:54:10.036743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:10.036743 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:54:10.037961 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:10.038461 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:10.038929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:10.038929 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:10.112196 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:10.112947 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:10.113626 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:10.114150 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:10.114534 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:10.114980 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:54:10.114980 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:10.114980 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:10.114980 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:10.114980 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:10.114980 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:10.114980 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:10.114980 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:10.116679 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:10.117439 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:14.807296 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:14.808524 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:14.841550 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:14.841982 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.843008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:54:14.843008 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:14.843008 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:14.843008 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:14.843008 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:14.843008 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:14.843008 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.843008 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:14.848687 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.849854 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:14.901526 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:14.902900 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:14.903197 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:14.903391 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.904241 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:54:14.904241 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:14.904241 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:14.904241 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:14.904241 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:14.904241 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:14.904241 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.904241 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:14.906425 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.907204 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:14.908258 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:14.908258 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:54:14.908258 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:14.908460 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.909088 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:54:14.909088 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:14.909088 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:14.909088 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:14.909088 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:14.909088 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:14.909088 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.909088 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:54:14.910493 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:54:14.911359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:14.911359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:54:14.911359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:14.911359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:54:14.911359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:54:14.911359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:14.911359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:54:14.911359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.911359 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:54:14.912154 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:14.912960 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:14.913956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:14.913956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:14.913956 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:15.032539 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:15.033048 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:15.033713 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:15.034094 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:15.034887 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:54:15.034887 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:15.034887 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:15.034887 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:15.034887 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:15.034887 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:15.034887 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:15.034887 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:54:15.036245 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:15.036984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:54:15.036984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:15.036984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:54:15.036984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:54:15.036984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:15.036984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:54:15.036984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:15.036984 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:54:15.037820 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:15.038329 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:15.038816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:15.038816 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:15.113087 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:15.113533 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:15.113925 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:15.114426 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:15.114562 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:15.115488 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:54:15.115488 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:15.115488 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:15.115488 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:15.115488 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:15.115488 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:15.115488 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:15.115488 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:15.116395 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:15.118007 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:19.807926 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:19.809590 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:19.844724 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:19.845048 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.845787 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Traceback (most recent call last):
Mar 23 22:54:19.845787 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:19.845787 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:19.845787 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:19.845787 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:19.845787 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:19.845787 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.845787 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-984dd979-0acc-4d41-a6a6-58a2a3938ec5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 781, in check_fdb_aging_settings
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:19.847008 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.847817 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:19.902454 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:19.903585 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-30981653-9cf2-456c-b4c8-6836a78525ee, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:19.903814 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-154c08d6-78c5-49fd-a785-0df3f4277a23, routes=[], if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:19.903976 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.904684 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Traceback (most recent call last):
Mar 23 22:54:19.904684 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:19.904684 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:19.904684 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:19.904684 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:19.904684 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:19.904684 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.904684 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-90336b3a-917c-4c98-8dab-525119820515 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 873, in check_router_default_route_empty_dst_ip
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:19.906556 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.907340 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:19.908148 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:19.908148 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Setting fip-distributed flag in NB_Global to False {{(pid=104232) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1072}}
Mar 23 22:54:19.908148 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:19.908350 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.908812 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Traceback (most recent call last):
Mar 23 22:54:19.908812 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:19.908812 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:19.908812 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:19.908812 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:19.908812 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:19.908812 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.908812 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:54:19.910129 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:54:19.910831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:19.910831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:54:19.910831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:19.910831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:54:19.910831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:54:19.910831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:19.910831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:54:19.910831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.910831 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [None req-36ecf95b-c088-4610-980c-dd8d53008682 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1076, in set_fip_distributed_flag
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 23 22:54:19.911889 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:19.912378 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:19.912878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:19.912878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:19.912878 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:20.033313 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:20.033603 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:20.034162 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:20.034440 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:20.035012 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:54:20.035012 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:20.035012 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:20.035012 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:20.035012 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:20.035012 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:20.035012 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:20.035012 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 23 22:54:20.036277 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:20.036785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 23 22:54:20.036785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:20.036785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 23 22:54:20.036785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 23 22:54:20.036785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:20.036785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 23 22:54:20.036785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:20.036785 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1131, in update_ha_failover
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 23 22:54:20.037858 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:20.038456 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:20.039018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:20.039018 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics 
Mar 23 22:54:20.113740 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=104232) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 23 22:54:20.114247 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=104232) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 23 22:54:20.114699 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:20.114941 np0000160382 neutron-ovn-maintenance-worker[104232]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=104232) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 23 22:54:20.115134 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:20.115723 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 23 22:54:20.115723 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:20.115723 np0000160382 neutron-ovn-maintenance-worker[104232]:     txn.results.put(txn.do_commit())
Mar 23 22:54:20.115723 np0000160382 neutron-ovn-maintenance-worker[104232]:                     ^^^^^^^^^^^^^^^
Mar 23 22:54:20.115723 np0000160382 neutron-ovn-maintenance-worker[104232]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:20.115723 np0000160382 neutron-ovn-maintenance-worker[104232]:     raise RuntimeError(msg)
Mar 23 22:54:20.115723 np0000160382 neutron-ovn-maintenance-worker[104232]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:20.115723 np0000160382 neutron-ovn-maintenance-worker[104232]: 
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics Traceback (most recent call last):
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     work()
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 84, in decorator
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 795, in update_mac_aging_settings
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:20.116895 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     next(self.gen)
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     with self.create_transaction(
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     self.result = self.commit()
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise result.ex
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 23 22:54:20.117993 np0000160382 neutron-ovn-maintenance-worker[104232]: ERROR futurist.periodics